A Content Originality Analysis of HRD Focused .../67531/metadc... · in retractions over a ten-year period starting in 1999. Understanding that not all retractions were plagiarism

APPROVED: Jeff Allen, Major Professor Kim Nimon, Co-Major Professor Lin Lin, Committee Member Cathy Norris, Chair of the Department of

Learning Technology Kinshuk, Dean of the College of

Information Victor Prybutok, Vice Provost of the

Toulouse Graduate School

A CONTENT ORIGINALITY ANALYSIS OF HRD FOCUSED DISSERTATIONS

AND PUBLISHED ACADEMIC ARTICLES USING Turnitin

PLAGIARISM DETECTION SOFTWARE

Robin James Mayes, B.S., M.S., M.S.

Dissertation Prepared for the Degree of

DOCTOR OF PHILOSOPHY

UNIVERSITY OF NORTH TEXAS

May 2017

Mayes, Robin James. A Content Originality Analysis of HRD Focused

Dissertations and Published Academic Articles using Turnitin Plagiarism Detection

Software. Doctor of Philosophy (Applied Technology and Performance Improvement),

May 2017, 167 pp., 15 tables, 19 figures, references, 167 titles.

This empirical exploratory study quantitatively analyzed content similarity indices

(potential plagiarism) from a corpus consisting of 360 dissertations and 360 published

articles. The population was defined using the filtering search criteria human resource

development, training and development, organizational development, career

development, or HRD. This study described in detail the process of collecting content

similarity analysis (CSA) metadata using Turnitin software (www.turnitin.com). This

researcher conducted robust descriptive statistics, a Wilcoxon signed-rank statistic

between the similarity indices before and after false positives were excluded, and a

multinomial logistic regression analysis to predict levels of plagiarism for the

dissertations and the published articles. The corpus of dissertations had an adjusted

rate of document similarity (potential plagiarism) of M = 9%, (SD = 6%) with 88.1% of

the dissertations in the low level of plagiarism, 9.7% in the high and 2.2% in the

excessive group. The corpus of published articles had an adjusted rate of document

similarity (potential plagiarism) of M = 11%, (SD = 10%) with 79.2% of the published

articles in the low level of plagiarism, 12.8% in the high and 8.1% in the excessive

group. Most of the difference between the dissertations and published articles were

attributed to plagiarism-of-self issues which were absent in the dissertations. Statistics

were also conducted which returned a statistically significant justification for employing

the investigative process of removing false positives, thereby adjusting the Turnitin

results. This study also found two independent variables (reference and word counts)

that predicted dissertation membership in the high (.15-.24) and excessive level (.25-

1.00) of plagiarism and published article membership in the excessive level (.25-1.00) of

plagiarism. I used multinomial logistic regression to establish the optimal prediction

model. The multinomial logistic regression results for the dissertations returned a

Nagelkerke pseudo R2 of .169 and for the published articles a Nagelkerke pseudo R2

.095.

ii

Copyright 2017

by

Robin James Mayes

iii

ACKNOWLEDGEMENTS

I would like to thank all of the professors, instructors, and administrators who

have personally taken an interest in my continuing education at the University of North

Texas. In particular, I would like to thank my Dissertation Committee. Professor Kim

Nimon has been instrumental in helping me focus on this topic, providing the editorial

and statistical support needed for the complexities required in corpus analysis. From

her, I gained valuable understandings of how to serve my future students in their

dissertation quest. Professor Jeff Allen has guided me with calming advice about the

difficulties that Ph.D. students encounter. Professor Lin has shared many service

opportunities with me, enabling me to learn about their importance to the University.

Moreover, I would like to acknowledge Professors Kinshuk, Cathy Norris, Mike

Spector, Mickey Wirschenski, Jerry Wirschenski, and John Turner for their ongoing

support, including sound advice for research, teaching, and service opportunities.

I would like to thank my Fiancée, Pamela McCleary, my daughters Polly

Pinneaux and Katie Minder, and my extended families (blood and in-laws) for their

patience. I often pushed aside personal obligations and activities while I focused on my

educational demands. I cannot say enough about the colleagues, friends, and family,

with whom I could share my experiences and perceived stresses. Their sacrifices are

hard to identify, but I appreciate that they enjoyed or at least tolerated this prolonged

professional and personal adventure.

iv

TABLE OF CONTENTS

Page ACKNOWLEDGEMENTS ............................................................................................... iii LIST OF TABLES ............................................................................................................ vi LIST OF FIGURES ......................................................................................................... vii INTRODUCTION ............................................................................................................. 1

Purpose and Rationale of Study ........................................................................... 4 Research Questions ............................................................................................. 5 Delimitations ....................................................................................................... 10 Limitations .......................................................................................................... 12

LITERATURE REVIEW ................................................................................................. 15

Plagiarism Definition, Categories, and Types ..................................................... 15 Factors Contributing to Plagiarism ...................................................................... 24 Consequences of Plagiarism .............................................................................. 27 Empirical Research ............................................................................................ 30 Reviewing Plagiarism Software .......................................................................... 43

METHODOLOGY .......................................................................................................... 47

Research Design Overview ................................................................................ 47 Population/Sample ............................................................................................. 50 Data Collection Process 1 .................................................................................. 54 Data Collection Process 2 .................................................................................. 56 Data Export Process ........................................................................................... 63 Data Collection Summary ................................................................................... 65 Data Analysis ...................................................................................................... 66

RESULTS ...................................................................................................................... 73

Dissertation Descriptive Statistics Results .......................................................... 73 Dissertation Differential Statistics Results .......................................................... 78 Dissertation Predictive Statistics Results ............................................................ 82 Published Article Descriptive Statistics Results .................................................. 88

v

Published Article Differential Statistics Results................................................... 93 Published Article Predictive Statistics Results .................................................... 96

DISCUSSION .............................................................................................................. 103

Discuss and Synthesize Research Findings ..................................................... 103 Descriptive Findings ......................................................................................... 104 Differential Findings .......................................................................................... 111 Prediction Findings ........................................................................................... 114 Discuss Document Similarity Levels ................................................................. 117 Issues and Obstacles ....................................................................................... 121 Conclusions ...................................................................................................... 125 Implications....................................................................................................... 128 Future Research ............................................................................................... 134

APPENDIX A: DISSERTATION FREQUENCY DETAIL TABLES ............................... 137 APPENDIX B: PUBLISHED ARTICLES FREQUENCY DETAIL TABLES .................. 140 APPENDIX C: SPSS AND R SYNTAX ........................................................................ 142 APPENDIX D: CSV FILES FIELD DESCRIPTIONS ................................................... 144 APPENDIX E: TURNITIN COA REPORT (NO ADJUSTMENTS) ............................... 147 REFERENCES ............................................................................................................ 148

vi

LIST OF TABLES

Page 1. Top Five Reported Retractions Rates for Plagiarism ............................................ 4

2. Comparison of Corpus Plagiarism Study Evaluation Levels ................................. 7

3. Variables Used for the Corpus Analysis in RQ1, RQ2 & RQ3 .............................. 8

4. Corpus-wide Descriptive Statistics for Dissertations ........................................... 73

5. Corpus-wide Spearman's rho Statistics for Dissertations ................................... 76

6. Descriptive Statistics for aDSI & aDSW for Dissertations by Groups ................. 77

7. Modeling MLR Analysis of Sampled Dissertations ............................................. 85

8. MLR Analysis of Sampled Dissertations ............................................................. 87

9. Corpus-wide Descriptive Statistics for Published Articles ................................... 89

10. Corpus-wide Spearman's rho Statistics for Published Articles ........................... 92

11. Descriptive Statistics for aDSI and aDSW for Published Articles........................ 93

12. Modeling MLR Analysis of Sampled Published Articles ...................................... 99

13. MLR Analysis of Sampled Published Articles ................................................... 101

14. Dissertation Corpus Plagiarism Studies Using Turnitin .................................... 107

15. Published Article Corpus Plagiarism Studies Using Turnitin ............................. 109

vii

LIST OF FIGURES

Page

1. Plagiarism category and type map ..................................................................... 16

2. Corpus plagiarism study process chart ............................................................... 48

3. Turnitin SSI interfaces for selecting SSI for examination and exclusion ............. 50

4. Random sampler software .................................................................................. 53

5. Adobe Acrobat JavaScript .................................................................................. 56

6. An example of a dissertation template similarity ................................................. 57

7. Flow diagram for verification of Turnitin content originality report ....................... 62

8. SRT Turnitin COA report data collection interface .............................................. 63

9. Spreadsheet formulas for double-checking SRT-DSI final adjustments ............. 64

10. Dissertation DSI & ADSI frequencies before and after adjustments ................... 79

11. Histogram exhibiting difference between dissertation paired DSI & aDSI .......... 80

12. Dissertation membership in document similarity levels ...................................... 82

13. MS Word VBA script for building all possible string subsets ............................... 84

14. Dissertation scatter plots for word count and reference count variables ............. 87

15. Article DSI & aDSI frequencies before and after adjustments ............................ 94

16. Histogram exhibiting difference between published article paired DSI & aDSI ... 95

17. Published article membership in document similarity levels ............................... 96

18. Published article scatter plots for word count and reference count variables ... 102

19. Trivial similarities of 6 words using a 10 word exemption ................................. 121

1

INTRODUCTION

Once Johannes Gutenberg invented the printing press in 1440, the stage was set

for a massive increase in the abuse of intellectual property rights, including plagiaristic

activities (Shelley, 2005). Moreover, the parallelisms of mass duplication and broader

distribution opportunities between the Gutenberg press and the global embracing of the

Internet further exacerbated the plagiarism problem (Chao, Wilhelm & Neureuther,

2009). Kock (1999) observed that the digitization and availability of documents for

global audiences had encouraged plagiaristic activities on a scale never experienced

before.

Cheung and Driver (2004) reaffirmed the existence of plagiarism and examined

its unintended consequences. They cautioned about plagiarizing existing knowledge

and advised, “Though researchers may examine the same topics from many angles and

in many populations, the scientific process is hindered when inquiries provide no new

contribution” (p. 7).

While academia promotes and expects quality publishing among its echelon of

researchers and professors (Kock, 1999), public universities facing reductions in state

and federally funded budgets have focused on increases in grant-funded research and

publishing (Shaw, 2002). Kock (1999) postulated that it is commonplace for universities

to reward young researchers or professors with promotions, pay increases, and offers of

tenure, greatly influenced by high publication counts. Moreover, academic publishing is

2

experiencing more student contributions. Hatch and Skipper (2016) examined 500

curriculum vitae from social science Ph.D. students and found that they had “averaged

4.3 peer-reviewed articles or book chapters before graduation” (p. 171). Shaw (2002),

O’Connor (2010), and Callahan (2014) maintained that these pressures to publish have

had its downside. They argued that increases in publishing pressures have diminished

the quality and importance of research and publishing activities. Baker (2015) affirmed

there were publishing quality problems when they reported that almost 50% of papers

they studied reported inaccurate statistical significance (p) values, some of which

directly affected the results of the published papers.

Karabag and Berggren (2012) further suggested that by reviewing the number of

retractions issued by publishers and authors, one could gauge the level of plagiaristic

activities. They reasoned that while institutions tended to shroud accusations and

resolutions of constituent plagiarism, publishers must publically retract manuscripts

when legitimate plagiarism issues were brought to their attention. In support of

Karabag's and Berggren's assertion of plagiarism and academic dishonesty, Cabral-

Cardoso (2004) discussed a bitter internal political struggle at a business school that

attempted to resolve an alleged accusation of plagiarism. He reported that:

In the case reported here, the informal rule appeared to be: “better keep things

quiet and out of public eyes” saving the university the embarrassment of having

to try to revoke an awarded degree and challenge some senior faculty. (p. 85)

Nevertheless, placing publication issues square in the public eye, SAGE

Publications retracted 60 articles from the Journal of Vibrations and Control (Retraction

Watch, 2014). The retraction notice stated that SAGE found a peer review ring created

3

by a single person who created multiple aliases using different email and SAGE user

accounts. While the notice did not list plagiarism as the driving force for the retractions,

there were serious fraudulent authorship issues that led to duplicate publication

submissions.

In another attempt to identify trends in article retractions, Wager and Williams

(2011) reported that the Medline medical literature had experienced a tenfold increase

in retractions over a ten-year period starting in 1999. Understanding that not all

retractions were plagiarism related, Decullier, Huot, Samson, and Maisonneuve (2013)

analyzed 235 retraction notices. They determined that while 28% of the articles were

retracted for mistakes, 20% were plagiarism related followed by fraud at 14%.

Amos (2014) studied retractions from the international biomedical field literature.

She searched for retractions that resulted from plagiarism, and duplicate publication

issues. Based on the findings from her exploratory study, she reported 20 national

affiliations in order by the number of all retractions counts for the period between 2008

and 2012. China, the United States, and India, all global economic powerhouses, round

out the top three places in her study. Amos (2014) concluded that:

Exploring plagiarism and duplicate publication across countries contributes to

understanding publishing and retraction practices. Only a very small percentage

of the published literature is ever retracted, and an even smaller percentage of

that literature is retracted because of plagiarism or duplicate publication …

However, these two reasons combined accounted for nearly 35% of all

retractions in the studied sample. (p.89)

4

Table 1 displays the top five national affiliation rankings of the 20 national affiliations

identified by Amos.

Purpose and Rationale of Study

The purpose of this study was to identify potentially plagiaristic activities by

examining corpora of dissertations and published articles focused on of human resource

development (HRD) using Turnitin content similarity analysis software. Swanson (1995)

defined HRD as “a process of developing and unleashing human expertise through

organization development and personnel training and development for the purpose of

improving performance” (p. 208).

Turnitin produces content originality reports or more descriptively correct, content

similarity reports. For the most part, these two terms are identical. While examining

content similarity indices and document descriptive data, this empirical study follows the

techniques and analytics similar to previous published empirical studies (e.g., Honig &

Bedi, 2012; Ison, 2012; Ison, 2014; Sun, 2013; Thomas & de Bruin, 2015).

Additionally, there is precedence of studying dissertations and published articles

for plagiarism. Ison (2012, 2014) examined plagiarism in dissertations in two related

Table 1

Top Five Reported Retractions Rates for Plagiarism and Duplicate Publication

National Affiliation Plagiarism Duplicate Publishing Totals

China 24 42 66 United States 17 26 43 India 18 7 25 Italy 16 2 18 Japan 2 13 15

5

studies. Honig and Bedi (2012), Sun (2013), Thomas and de Bruin (2014), and others

have examined plagiarism in published articles.

The rationale for this HRD-focused corpora plagiarism study parallels with what

Honig and Bedi (2012) reported as their underlying reasons for their corpus plagiarism

study in the discipline of management and administration:

We strongly believe this study shows the need to identify and verify the originality

of scholarship and should be an increasingly important responsibility of the

Academy of Management. It is our hope is that this study leads to the

development and implementation of specific screening systems, as well as more

repetitive and transparent ethical guidelines, in order to enhance the scholarship

standards represented by the Academy of Management. (p. 118)

Moreover, this study adds to the literature by identifying and describing the

process of investigating plagiarism, using Turnitin on existing documents, and

answering the following research questions.

Research Questions

For this exploratory quantitative study, I considered three research questions for

each corpus (dissertations and published articles) based upon Turnitin content similarity

results and document metadata. Turnitin defines content similarity results as data

collected from a document using a plagiarism detection software system (i.e., Turnitin).

Rouse defined document metadata as “information attached to a text-based file that

may not be visible on the face of the document” (2014, p. 1)

The Turnitin content originality report provided various related content similarity

data. The document similarity index (DSI) and the document similarity word counts

6

(DSW) were the starting points. The DSI was the amount of the document’s total

content similarity that Turnitin had identified with other documents in its document

collection database before any exclusions or adjustment were applied. The DSW was a

synthesized value derived from the document word count times the DSI (a percentage).

I adjusted the DSI using “qualitative judgments” for excluding “false text-matching

incidences” of source similarity indices (SSI) from the calculations (Sun, 2013, p. 267).

The SSI-substantive types referred to in the research questions are the SSI equal or

larger than 5%. I recorded the adjusted DSI as the aDSI. The adjusted document

similarity word (aDSW) count was a synthesized value derived from the document word

count times the resulting aDSI (a percentage).

The document metadata collected were document research method (quantitative,

qualitative, mixed, or other), year of publication, author count, word count, and

reference count. As each document was selected, I collected this metadata information

during the document review. I recorded the collected data in the Scholarly Research

Tracker (SRT; Mayes, 2016) database.

The study examined many of the different evaluation strategies and incorporated

various features from all of them (see Table 2). There were several examples of

plagiarism evaluation categories using two and four level criteria. This study employed

the labels “Low,” “High,” and “Excessive” which I partially derived from a combination of

Thomas and de Bruin’s (2014) labels. This study also used Thomas and de Bruin’s

(2014) rate levels because they based them upon the aDSI (overlaps and false

positives removed). For the first level, the first two rates were combined: “Low” was 0%-

14%, “High” was 15%-24%, and “Excessive” was 25%-100%.

7

Bedeian (2014) emphasized the importance that descriptive statistics afford a

researcher in a basic understanding of primary data collected in a quantitative study.

The first set of research questions channeled the research toward a statistical

description of the corpus of sampled dissertations and published articles employing

distributions, groupings, categories, and measures of central tendencies. See Table 3

for a list of variables. Moreover, the SSI-substantive types referred to in the research

questions are the SSI equal to or larger than 5%. These SSIs were descriptively

Table 2

Comparison of Corpus Plagiarism Study Evaluation Levels

Authors Year Variable L1 L2 L3 L4

Turnitin 2017 DSI Green 0-24%

Yellow 25%-49%

Orange 50%-74%

Red 75%-100%

Mayesa 2017 aDSI Low 0-14%

High 15-24%

Excessive 25%-100%

Thomas and de Bruin 2014 aDSI Low 0-9%

Moderate 10%-14%

High 15%-24%

Excessive 25%-100%

Zhang & Jia Survey Resultsb

2012 OSI (DSI) Minor 8.99%

Moderate 21.69%

Serious 38.78%

Rejection 50.49%

Masic 2012 Non Original Acceptable 0-24%

Rejected 25%-100%

Walker 2010 DSI Moderate 0-19%

Extensive 20%-100%

Batane 2010 aDSI Legitimate 0%

Low 1%-34%

Medium 35%-69%

High 70%-100%

Bretag and Mahmud 2009 DSI 0-10% 11%-24%

Higher Education Commission, Pakistana

n.d. University Guidelines for Turnitin

Acceptable 0-18%

Rejected 19%-100%

aSet SSI Revision Rate at 5% bUsed the DSI evaluation rates for evaluating the SSI rates.

8

identified as the potential of plagiarism-of-other or plagiarism-of-self.

RQ1.1: What are the descriptive statistics of Turnitin’s reported document similarity

indices (DSI), including percentages and synthesized word counts; researcher-adjusted

document similarity indices (aDSI), including percentages and synthesized word counts;

source similarity indices (SSI-substantive type), including percentages and synthesized

word counts; and document metadata for the corpus of sampled dissertations?



Table 3

Variables Used for the Corpus Analysis in RQ1, RQ2 & RQ3

Variable Name RQ Scale Values

Document Similarity Level (DSL) [DV] 3 Polychotomous Low, High or Excessive

Year of Publication (YOP) [IV] 1, 3 Continuous 2011-2015

Research Method (DRM) [IV] 1, 3 Categorical Quant, Qualt, Other

Author Count (ACT) [IV] 1.2, 3.2 Continuous Greater than 0

Word Count (WCT) [IV] 1, 3 Continuous Greater than 0

Reference Count (RCT) [IV] 1, 3 Continuous Greater than 0

Document Similarity Index (DSI) [DV] 1, 2 Continuous 0-100%

Document Similarity Word Count (DSW) 1 Continuous Greater than 0

Adjusted Document Similarity Index (aDSI) [DV] 1, 2, 3 Continuous 0-100%

Adjusted Document Similarity Word Count (aDSW) 1 Continuous Greater than 0

Substantive Other SSIa 1 Continuous Frequency

Substantive Self SSIa 1 Continuous Frequency

Substantive Other SSI (Mean Similarity)a Continuous 5-100%

Substantive Self SSI ( Mean Similarity)a 1 Continuous 5-100%

Substantive Other SSI (Mean Word Count)a 1 Continuous Greater than 1

Substantive Self SSI (Mena Word Count)a 1 Continuous Greater than 1

aSubstantive SSI are equal to or greater than 5%

9

document similarity indices (aDSI) including percentages and synthesized word counts;


word counts; and document metadata for the corpus of sampled published articles?

Batane (2010) found that Turnitin suffered from a “tendency of the software to

identify the material as plagiarized” (p.3). He suggested that Turnitin users verify all

instances of identified content similarities and make the necessary adjustments. This

study provided an opportunity to identify the statistical and practical significance of the

required adjustments or corrections. In simpler terms, RQ2.1 and RQ2.2 ask the

question “is it necessary for a plagiarism researcher to verify the results of a Turnitin

analysis report?

RQ2.1: Are there statistically and practically significant differences between the levels of

Turnitin’s reported document similarity indices (DSI) and my adjusted document

similarity indices (aDSI) for the corpus of sampled dissertations?



similarity indices (aDSI) for the corpus of sampled published articles?

Often corpus-based plagiarism studies analyze documents for evidence of

plagiarism and include predictive analytics based upon various researcher available

predictor variables against the measured plagiarism values (e.g. Honig & Bedi, 2012;

Ison, 2012; Perfect, Defeldre, Elliman & Dehon, 2011; Sun, 2013; Thomas & de Bruin,

2014). However, there are few actionable research outcomes based on the findings.

The profession should conduct additional research on an ongoing basis. This study

includes a predictive research component.

10

RQ3.1: Does document research method, year of publication, word count, and

reference count predict membership in low, high or excessive levels of the plagiarism

categories for the corpus of sampled dissertations?

RQ3.2: Does document research method, year of publication, author count, word count,

and reference count predict membership in low, high, or excessive levels of the

plagiarism categories for the corpus of sampled published articles?

Delimitations

Delimitations set a study’s boundaries (Simon, 2011). The first delimitation was

the selection of two widely commercially accepted document databases (EBSCO and

ProQuest Dissertation and Theses). My reliance on EBSCO and ProQuest was critical

to my study. However, from the researchers’ standpoint, Turnitin’s internal functionality

and accuracy are undocumented. Moreover, Turnitin’s document acquisition process

and the schedule of document availability for researchers was unknown. For example,

ProQuest dissertations, by author choice, can remain unavailable to queries for up to

five years. I also had found that most EBSCO query results would report a larger

number of documents than were listed and retrievable. For example, a user query might

retrieve ten pages and 120 document. Upon review, as a user reached the eighth page,

there were no more documents to review and far less than 120 documents. Given these

potential anomalies, during the engineering phase of this study, I secured the document

population on which I would later build my sample. I limited the corpora to the publishing

years 2011 through 2015 as described in the methodology section, securing a

document population file through the end of the 2015 calendar year.

11

I selected Turnitin software for the content originality analysis because Turnitin is

one of the leading COA software applications. However, Turnitin only identifies content

similarities and cannot distinguish between the different categories and types of

plagiarism (see Figure 1, p. 16). When Turnitin could provide information that would

lead to a reasonable differentiation between substantive plagiarism-of-others and

plagiarism-of-self, I recorded that evidence.

The practice of reverse plagiarism is when an author gives another author credit

when none is warranted (Jent, 1967; Knight, 2013; Moten, 2014). Turnitin had no

practical way of detecting reverse plagiarism. This study does not include reverse

plagiarism in its calculations and analysis.

I configured Turnitin’s settings to remove as much non-material content similarity

as possible. I excluded quoted content and references, word strings less than ten

words, and student papers from the analysis (cf. Thomas & de Bruin, 2015). I manually

removed major publisher template similarities like repeated copyright notifications from

the documents, before submission to Turnitin. These pieces of text frequently show up

in Turnitin reports in significant numbers of SSI that erroneously affect the DSI.

Document format and size were areas of concern. A Turnitin COA report on a

PDF that is in image format was not possible. I converted documents in image format to

text format using Adobe Acrobat’s OCR capabilities before submission. However, there

was one document, which contained many images of the author's hand-written notes as

content. Turnitin did not include these images in the COA report.

Turnitin had a submission limitation of 400 pages. I submitted one document

larger than 400 pages in parts. I removed document pages not requiring an originality

12

check from the largest documents to reduce pages below 400. These document

preparation and analysis procedures were consistent, despite variances in document

type, author, word, reference counts, and research method employed.

The research revealed there was much concern about plagiarism by English-as-

a-second-language (ESL) authors (Sun, 2013). This phenomenon was beyond the

scope of this study because obtaining ESL demographic data would be difficult.

Moreover, this study did not identify plagiarism in language translations. Whether this is

a case of plagiarism-of-others or plagiarism-of-self, articles translated from one

language to other languages are difficult to process with Turnitin. While in its infancy,

researchers are continuing with the development of improved language translators that

may lead to better identification of cross-language content similarities (PT, 2011).

Limitations

Limitations are the potential shortcomings or weakness in the design of a study

(Simon, 2011). Recognizing the first limitation (page 11), I collected a random sample of

documents, predominately selected from the human resource development (HRD) field

using keyword-filtering technologies. These document searches returned various cross-

disciplinary documents that were included in this study. However, I do not generalize

this study's results beyond the HRD field.

Another limitation is that I conducted this study over a short period, and my

readers should not generalize the results as possessing any longevity beyond the

current strategies and technologies. Nor could I control the population counts within

each subpopulation. I experienced a tendency in the ProQuest and EBSCO to have

decreasing subpopulations in the later periods within a queried range. As a follow-up

13

examination, I queried both databases for the years 2012-2016 (an increase in one

year) in January 2017 and found the year 2016 had less than 1/2 the document count

as compared with each of the other individual years (2012-2015). One might conclude

that document database publishing regularly suffers collection, verification, and

processing delays.

The reliance on the features and accuracy of Turnitin, including the diversity of its

document collection database, was another limitation. Turnitin only identified text

similarities between the submitted document and the available documents located in its

document collection database (Mulcahy & Goodacre, 2004). Turnitin cannot include all

articles and textbooks in the Turnitin document collection database. Therefore, the

Turnitin report process can lead to Type 1 Errors or the failing to identify potential

content similarities (cf. Stevens, 2009). Moreover, while Turnitin can remove quoted

materials from the COA, Turnitin does not always do so, thus leading to Type 2 Errors

or falsely identifying content similarities (cf. Stevens, 2009). To clarify, for the remainder

of this study, a false positive (Type 2 Error) refers to content similarities that are not

evidence of plagiarism. A common example of a false positive is a publisher inserting a

notice in the document; Turnitin may identify the content of the notice as plagiarism

across many documents with the same notice.

Turnitin’s Parent company, iParadigms, engineered Turnitin to analyze

unpublished or recently published manuscripts. The COA process becomes complex for

previously published documents with copies and pieces of those documents spread

throughout the Internet. I found that Turnitin identified three types of source similarity

indices (SSIs): Publication, Internet source, and Student paper. The publication SSIs

14

were easy to investigate if Turnitin provided a direct link to the source document. If the

document collection database is the only source, it was difficult to read and could not be

searched or select and copied. The Internet source SSIs also provided documents to

validate similarities. However, text similarities in these documents were often trivial and

template based. Text retrieved from web pages such as, but not limited to, document-

abstracts and keywords were numerous. The job of validating similarities was often

difficult because of a lack of listed author names and publication dates. The SSIs

identified as Student papers were the most difficult to validate. If the students or

instructors have submitted these documents to Turnitin, the submissions were placed in

a separate document collection database (DCD) apart from the published article DCD.

However, Turnitin would not make the documents available for inspection without

permission from the person who submitted them to Turnitin. This study followed

Thomas and de Bruin (2015) and removed the student document collection database

from consideration in the content originality report as these documents were not

published or copyrighted

Another limitation is Turnitin’s sole reliance on percentages. Should a 100,000-

word document with 6% similarities (6000 words), be compared with a 10,000-word

document with 20% similarities (2000 words)? The question begs to be asked, “Which

document has the more serious plagiarism problem. As Ison (2012) noted:

The size of dissertation [document] must be considered … when examining

similarity indices, as even a 2% overlap of a 200-page dissertation [document]

essentially means there are four pages worth of unoriginal material. (p.234)

15

LITERATURE REVIEW

Plagiarism Definition, Categories, and Types

The Committee on Publication Ethics (COPE) formed in 1977 to address a lack

of formal guidance with which to handle unethical research and publishing conduct

(2014). A group of biomedical journal editors associated with COPE instigated a code of

conduct guidelines. Using the COPE guidelines, Hulten, Nicholls, Winslet and Kmiot

(2000) indicated that “plagiarism ranges from the unreferenced use of others' published

and unpublished ideas to submission under the new authorship of a complete paper,

possibly in a different language” (p. 247). However, in research of the literature,

plagiarism appears to be more complex and difficult to apply conceptually than what

Hulten et al. (2000) implied. According to a review of the literature, there are three main

categories of plagiarism: plagiarism [plagiarism-of-others] (O'Connor, 2010), self-

plagiarism [plagiarism-of-self] (Cheung & Driver, 2004; Yentis, 2010), and reverse

plagiarism (Jent, 1967; Knight, 2013; Moten, 2014). For the duration of this dissertation

and the purpose of adding a level of clarity, the term “plagiarism” will encompass all

three categories of plagiarism as shown in Figure 1. I used Turnitin’s text-similarity

detection system to identify all of the plagiarism types as identified as green in Figure 1.

Moreover, while Turnitin can detect content similarities leading to potential evidence of

plagiarism-of-others and plagiarism-of-self, it is unable to differentiate between the two.

Moreover, Turnitin cannot identify the types within the three categories. Any

identification beyond text similarities is left to an investigator and often subjective

decisions based upon a preponderance of the evidence.

16

Plagiarism-of-Others

Handa (2008) defined plagiarism-of-others as:

The failure to acknowledge other colleagues’ scientific work - their ideas,

language, or data. It [plagiarism-of-others] may include verbatim copying of

passages without citing the original contributor, rewording of ideas, paraphrasing,

Figure 1. Plagiarism category and type map. Using Turnitin this study identified potential overall plagiarism (blue box) by examining content similarities that included, but not limited to categories of substantive plagiarism-of-others and substantive plagiarism-of-self (green boxes). However, this study does not identify or summarize plagiarism types within each category (yellow boxes). Moreover, Turnitin cannot directly identify reverse plagiarism, data redundancy and layered citations (red boxes).

17

and even total reproduction by simply changing the authors’ names and trying to

pass the material as one’s own. (p. 301)

According to O’Connor (2010), plagiarism-of-others is a common publication

problem. There are various types of plagiarism-of-others (i.e., document, paragraph,

paraphrasing, and layered citations). Document plagiarism-of-others occurs when a

substantial, if not all of a document is presented as one’s own work. For example,

McMurtry (2001) found that students used web-based services (paper-mills) to buy

documents to be passed off as their own. She reported instances of students who had

paid as much as $35.00 a page for documents from web-based paper mills. However,

as Posner (2007) indicated, that kind of transgression would be better defined as an

academic fraud, not plagiarism. A student's intent is not to wrong the original author, but

to deceive the reader (faculty). However, there remains a question about who owns the

copyright should the students decide to publish their work.

Paragraph plagiarism-of-others is when one copies paragraphs or sentences

directly into one’s own work without proper citations. Chao, Wilhelm, and Neureuther

(2009) posited that powerful document databases such as Google Docs, EBSCO,

LexisNexis, and ProQuest provide researchers seemingly unlimited resources of

electronic text. The digital age has affected paragraph plagiarism-of-others with simple

“copy and paste” functionality (Chao et al., 2009; O'Connor, 2010). Paraphrasing is an

attempt to correct or hide paragraph plagiarism-of-others (Chao et al., 2009).

Inexperienced authors may copy and paste another’s work into their documents and

then change several of the words believing that this infraction is technically not

plagiarism-of-others. At a minimum, an author must provide a citation and reference as

18

the original idea remains another’s work. If the majority of words from the original

source remain, quotations are additionally required. If an author inadequately

paraphrases copied content, CSA software will still identify the similarities (Chao et al.,

2009). Whether to count poorly executed paraphrasing with citations, but without

quotation marks, as plagiarism is a subjective call made by the investigator.

Layered citations or citation overlaps can also cause plagiarism-of-others issues

even for the most experienced author. Tucci and Galwankar (2011) reported that a

quote is often credited to the publication from which the text was retrieved, but prior

unearthed documents may have been the original source. When citing one’s source, a

researcher must execute due diligence in identifying prior publications where that

content might have first originated (Tucci & Galwankar, 2011). Researchers often use

the term "snowballing" to describe how references often lead to other related references

(Regmi & Naidoo, 2013, p.33). This process often reveals prior and sometimes original

sources. However, the APA (2010) stated that if one does not have access to older

works one must still include the reference, but may cite the source using this format “as

cited in” (p. 178).

Plagiarism-of-Self

One of the most misunderstood and least researched types of plagiarism is self-

plagiarism, which this study refers to as plagiarism-of-self. The APA (2010) identified

plagiarism-of-self as the “practice of presenting one’s own previously published works

as though it [previously published works] were new” (p. 170). Duplicate publication

(including translations), text recycling, and data redundancy are three types of

plagiarism-of-self (cf. Adhikari, 2010; Cheung & Driver, 2004; Roig, 2010; Yentis, 2010).

19

Duplicate publication often occurs when authors submit the same manuscript to

multiple journals (Cheung & Driver, 2004; Yentis, 2010). Susser and Yankauer (1993)

reported that while the inclusion or order of authors may have been changed, and the

article may have had some minor changes in the wording, duplicate publications are the

same work, just repackaged.

Language translations of prior works are also considered plagiarism-of-self

(Cheung & Driver, 2004). However, Yank and Barnes (2003) found that in practice,

about 30% of surveyed editors and authors felt that redundant publishing in a non-

English journal was acceptable. Sibbald (2000) added that duplicate publishing is often

critical for reaching diverse audiences. However, Sibbald cautioned that the copyright

holder must provide a release and the secondary publisher must be aware of the

previous publication (2000).

An often-misunderstood instance of duplicate publishing is the author(s) article

having been published in conference proceedings; then later submitted for publication in

a journal. If conference proceeding are publicly accessible or the authors relinquish their

copyright protection, the article cannot be published again without documented

permissions and notifications according to Sibbald (2000). Callahan (2012) warned

authors that they should:

Be vigilant about ensuring that your intellectual property is not openly accessible

online if you want to continue publishing on the topic. This is especially important

for working on conference papers you hope to publish as chapters or journal

articles. Another option is to ensure that, if you want to publish your work later,

20

you do not submit full papers to conferences, but instead submit only abstracts.

(p. 8)

The APA (2010) uses the phrase “limited circulation” as a keyword in

understanding duplicate publishing issues (p.13). Governmental agency studies,

University Department reports, and U.S. dissertations are not normally marketed or

available to the public on a broad scale, thus can be republished in modified or

extended form. However, conference proceedings and book chapters that are offered to

the public are not considered in limited circulations and cannot be republished in whole

or part. Regarding APA brief reports, the APA stated that if they “include sufficient

descriptions of methodologies to allow for replications; the brief report is the archival

record for the work” (p.13). The brief, or variation of, cannot be republished without

publisher permissions and notification to the readers. Furthermore, the APA states if a

“brief report is published in an APA journal it is with the understanding that the extended

report will not be published elsewhere.” That may imply that this type duplicate

publication is acceptable if it is published within the same publishing house. The APA

does not say that other publishers may allow duplicate publication within their publishing

house.

According to Adhikari (2010) and Roig (2010), another form of plagiarism-of-self

occurs when authors use parts of their previous works in current works without proper

citations. They both referred to this type of plagiarism-of-self as “text recycling”

(Adhikari, 2010, p. 77; Roig 2010, p. 299). Often researchers start repeating

themselves, especially after publishing multiple manuscripts, using paragraphs and

even whole sections from their previous works (Adhikari, 2010; Roig, 2010). While

21

authors must be diligent in their disclosure of previously published ideas, APA (2010)

specified that if self-citation is awkward, an author could limit the depth of the citations.

A simple phrase (e.g., “as I have previously discussed”) often suffices (APA, 2010, p.

16). Moreover, APA (2010) deemed text recycling to describe analytical approaches as

acceptable. Robinson (2014) argued that while plagiarism-of-self issues in the

biomedical field can be a serious matter, in other academic fields “institutions . . . should

not be rushing to incorporate strictures against all forms of textual recycling into their

academic integrity policies” (p. 275).

Data redundancy is another plagiarism-of-self (O’Connor, 2010; Yentis, 2010).

Analyzing subsets of data collected at the same time and reporting the results across

multiple studies and manuscripts may be considered a form of plagiarism-of-self. Terms

like “data salami slicing,” “data subdivision,” or “fragmented publication” are used to

describe this type of unethical research (e.g., Adhikari, 2010; Farthing, 2006; Karlsson &

Beaufils, 2013). Even if a study is analyzing different variables, using the same

database may be considered data redundancy. If there are any potential ethical

considerations, the author must provide an author’s note to the editor and readers.

According to APA (2010):

Data that can be meaningfully combined within a single publication should be

presented together to enhance effective communication . . . Authors must inform

the editor of any similar manuscript . . . Authors have a responsibility to reveal to

the reader that portions of the new work were previously published. (p. 14-15)

22

In itself, data redundancy may seem impossible to detect. However, Spielmans,

Biehn, and Sawrey (2010) found data redundancy plagiarism issues by identifying text

similarities across content using searches by topic, authors, and institution names.

There are much confusion and disagreement on how serious of a publication

problem plagiarism-of-self is. Established authors are often pitted against less

experienced authors trying to establish a publishing reputation. Callahan (2014) advised

HRD colleagues to:

Resist the urge to use the label self-plagiarism. Such a label facilitates a moral

panic that is unjustified and unjust . . . There is much to question about the extent

to which self-plagiarism is a real issue or a manufactured issue to serve the

interests of a selected few. (p. 7-8)

Along with a similar vein, Robinson (2014) concluded that while plagiarism-of-self

“misrepresents the nature and size of the author’s accomplishment” and potentially

fosters resentment among one’s colleagues, plagiarism-of-self “should not be treated as

an academic integrity issue” (p. 270).

However, Schminke and Ambrose (2014) authored an editorial explanation to the

retraction of one of their previous editorials. During activities of a doctoral class, the

students noticed an editorial by Schminke and Ambrose had considerable similarities

(26%) with another paper written and published by Schminke. Their instructor notified

the editors of the Academy of Management Review of the evidence of plagiarism-of-

self. Given the lack of a predefined course of action, the editors decided on a remedy

that included an article retraction and the authors editorializing on what happened and

why. An interesting aside was that the editors asked the students if this remedy was

23

sufficient. The students replied they were satisfied. The lesson learned from this case is

that any interested person, including students, can bring up even a renowned professor

or author on charges of plagiarism. Any professor or author could potentially find

themselves in a situation with career-damaging consequences (cf. David, 2011).

While students do not normally publish their assignments, Halupa (2014)

identifies student plagiarism-of-self as the result of students recycling their work across

multiple courses. There is some debate whether text recycling of unpublished content is

a plagiarism of any type. It is common practice for graduate students to build upon their

previous work; recycling has not been a serious issue. Moreover, citations in theses or

dissertations are not required for original work from a student’s previous unpublished

assignments. However, depending on an institution's academic policy, this type of

infraction could be an attempt to deceive the institution and faculty.

Reverse Plagiarism

Reverse or inverse plagiarism is the attribution of one’s own ideas to another

author (Jent, 1967; Moten, 2014). While the motives may seem beyond comprehension,

students have used reverse plagiarism to increase their reference counts (Greenbird,

2009). Authors have also attempted to add additional credibility to their works by

associating an idea with a more renowned author (Turmfalke, 2010). Shanmugam

(2009) studied trainee teachers’ assignment work looking for citation errors and found

that 16.01% of the citation errors were incorrect author assignments. While

Shanmugam did not identify these errors as reverse plagiarism, the evidence does

potentially support reverse plagiarism (2009).

A variation of reverse plagiarism is “editing as reverse plagiarism” (Knight, 2013,

24

n. p.). Knight reported he found that the editor had significantly changed a manuscript

which he had authored and submitted for publication. Furthermore, the publisher had

published the edited manuscript without his review. Knight felt uncomfortable in having

the edited version attributed to him and complained. The editor only offered an apology

for potentially hurting his feelings. Reverse plagiarism is extremely difficult to detect

even at the reviewing stage. Often instances of reverse plagiarism are only called to

attention by the authors, who had been erroneously cited.

Factors Contributing to Plagiarism

A review of the relevant literature (e.g., Fang, Steen, & Casadevall, 2012; Horner

& Minifie, 2011; Onwuegbuzie & Daniel, 2005) suggested that misconceptions, mistakes

or errors, fraud, and cultural differences are the primary factors contributing to

plagiarism. Samuelson (1994) added that “the sin of laziness” is at the heart of

plagiarism (p. 24). He stated if an author just rewrote their prose, they could avoid most

instances of plagiarism (Samuelson, 1994).

Misconceptions

The most common reason for plagiarism violations appears to be author

ignorance (Horner & Minifie, 2011). College students and sometimes their faculty are

not knowledgeable about what constitutes plagiarism. Often they have not received any

formal education on, nor any extensive experience with applying publishing ethics (e.g.,

Cheema, Mahmood, Mahmood & Shah, 2011; Marcus & Beck, 2011; Orim, Davies,

Borg & Glendinning, 2013). Moreover, variations in industry citation and reference

standards, often documented in multiple publishing guides, exacerbate the problem

(Auer & Krupar, 2001). Auer and Krupar (2001) found students became confused when

25

exposed to both APA and MLA citation formats. Considering that the EBSCO

publication database commonly provides seven different citation formats, one can

understand the difficulty in applying citation and reference standards. Furthermore, one

can conclude that few students understand copyright laws, fair use doctrine, and

potential litigation entanglements. Chao et al. (2009) demonstrated that students, who

have received instruction in plagiarism avoidance, were significantly less likely to

plagiarize.

Mistakes, Errors or Common Exceptions

Onwuegbuzie and Daniel (2005) indicated that the complexity of bringing a

manuscript to a publishable level could create instances of citation and reference errors.

Sometimes an author just forgets that they were not the source of an idea and failed to

cite the contribution. The term cryptomnesia as defined by Hege (2008) illustrates this

common plagiaristic mistake:

[Cryptomnesia] inadvertent plagiarism represents a memory failure in which

individuals either misattribute the source of the information to themselves rather

than to the true originator or they simply do not recall having encountered it

before and claim that it is their own novel creation. (p. ii)

However, the APA (2010) stated that to credit the cited source; citations and

references must be complete and correct. While these kinds of errors may not seem

serious, the publishing profession discourages citations that cannot direct the reader to

the cited work (Onwuegbuzie & Daniel, 2005). In general, the reader must have the

opportunity to check the accuracy of the source the author used.

26

There are areas where plagiarism is accepted, or at least remains unchallenged.

The writing of textbooks is a publication area where the reader has accepted that little in

a textbook is of original ideas and that if every sentence were cited, the book would be

unreadable (Posner, 2007). He posited that most readers of textbooks understand that

the content is a compilation of the topic as accumulated over time and for the most part,

are uninterested in knowing the originality of the various concepts.

Fraud

Posner (2007) advocated that concealment (fraud) is an essential characteristic

of plagiarism. Searching for evidence that fraud exists, Fang et al. (2012) studied 2,047

retracted biomedical and life-science research articles. They posited that publishers

retracted 43% of the articles because of fraud or suspected fraud. While rare, an author

may commit fraud and intentionally attempt to pass off another’s product as their work

(Fang et al., 2012). In areas where potential plagiarism occurs, issues of ownership and

copyrights are often subject to litigation in civil court (Mawdsley, 2009; Posner, 2007).

Cultural Differences

Occasionally authors from one culture engage in what other cultures consider

plagiarism. There is evidence that not all cultures prescribe to the same ethical

standards regarding publication (Shi, 2006). Shi (2006) further explained that authors

whose second language is English might feel the need to engage in “textual

appropriation” or borrow words and paragraphs as they attempt to master the language

(p. 264). However, Liu (2005) rebuked the idea that plagiarism is an acceptable practice

in his Chinese culture. Liu described his educational experiences in China as having

never subscribed to the idea that plagiaristic activities were acceptable (2005)

27

Consequences of Plagiarism

Plagiarism is not a victimless crime and affects many stakeholders in the

research and publication profession. Authors, institutions of higher education,

corporations, editors, and publishers all have a stake in quality research and

publications. One's plagiaristic activities can appropriate profits by the taking of

copyrighted materials and then claim the spoils as from one’s own work (Posner, 2007).

However, Posner stated:

Though there is no legal wrong named “plagiarism,” plagiarism can become the

basis of a lawsuit if it infringes upon a copyright or breaks the contract between

author and publisher. (p. 34)

According to literature (e.g., Hendee, 2007; Karlsson & Beaufils, 2013; Neville &

Wadler, 2005), there is a wide variety of consequences for those engaging in plagiaristic

activities. Authors should consider the potential ramifications, ranging from a simple

article rejection to a more serious article retraction before engaging in unethical

publishing tactics. A founded accusation of plagiarism can discredit the author and can

lead to serious career-damaging consequences such as dismissal (Neville & Wadler,

2005). Moreover, accusations of plagiarism can lead to entanglement in legal

proceedings (Kock, 1999; Posner, 2007; Rubio, 2013).

Article Retraction

Article retraction is the most visible consequence of plagiarism when the

redundant publication is the cause (Karlsson & Beaufils, 2013). Amos (2014) studied

retractions in the biomedical literature. She reported that out of 754 retractions about

one-third of them (253) were from plagiarism (130) and duplicate publication infractions

28

(123). She also captured the authors’ national affiliation and created a top 20 national

affiliation ranking by retractions counts. See Table 1 for a national affiliation ranking of

the top five of her 20 by plagiarism and duplicate publication infractions.

SAGE Publications retracted 60 articles from the Journal of Vibrations and

Control (Retraction Watch, 2014). The retraction notice stated that SAGE found a peer

review ring created by a single person creating multiple aliases using different email and

SAGE user accounts. While the notice did not list plagiarism as a driving force of the

retractions, there were serious fraudulent authorship issues that led to duplicate

publication submissions.

Karlsson and Beaufils (2013) posited that publication retractions could critically

reflect on the reputation of publishers and their editors. Moreover, disseminating

retraction notices is a complex process as publishers must notify international

databases and provide justifications for retractions. Furthermore, a retraction can lead

to the death of a publishing career. An institution can place an author in censorship

status, and an affected publisher could ban the author(s) from future submissions

(Karlsson & Beaufils, 2013; Wittmaack, 2005).

Legal Proceedings

Hendee (2007) stated a major consequence of any engagement in plagiaristic

activities is often entanglement in costly legal proceedings. A publisher who owns the

copyrighted material may engage in litigation against a plagiarizer, even when the

plagiarizer is the original author. Cheung and Driver (2004) warned, “redundant

publication or self-plagiarism [plagiarism-of-self] can constitute copyright infringement if

authors reuse text or elements of papers that they no longer own” (p. 6). However,

29

Cheung and Driver (2004) concluded that the U.S. legal system remains sympathetic to

those who have reused their work and often rules in their favor.

Probably the most serious consequence resulting from plagiaristic activities is the

chance of entanglement in criminal proceedings (iThenticate, n.d.). For example, if

federal research grants are involved, and the research leads to a misuse of funds,

criminal proceedings may be involved. The Inspector General Act of 1978, as Amended

authorized the Federal Office of Inspector General the authority to investigate and

recommend prosecution of cases where grant recipients have misused government

funds, including research grants tainted by research misconduct (U.S. Government,

2014). The Office of Inspector General reaffirmed these efforts in their Semiannual

Report to Congress (2014) which stated:

Research misconduct damages the scientific enterprise, is a potential misuse of

public funds, and undermines the trust of citizens in government-funded

research. It is imperative to the integrity of research funded with taxpayer dollars

that NSF-funded researchers carry out their projects with the highest ethical

standards. For these reasons, pursuing allegations of research misconduct

(plagiarism, data fabrication, and data falsification) by NSF-funded researchers

continues to be a focus of our investigative work. (p. 21)

Moreover, the entanglement of unethical research with the legal system extends

beyond the United States. Rubio (2013) and Cromwell (2012) reported on a Columbian

Supreme Court decision, which sentenced a professor in May of 2010 “to two years in

prison plus monetary and civil sanctions for plagiarizing a student’s thesis” (Rubio,

2013, p. 141).

30

Tainted Research

Still an important, yet less visible consequence is that all categories of plagiarism

often misrepresent the influential weight of research (Cheung & Driver, 2004). Cheung

and Driver (2004) explained that duplicate conclusions coming from plagiaristic activities

make findings more credible than they should be. On the other hand, data redundancies

(salami slicing) tend to dissect knowledge into pieces as opposed to fitting the

discoveries together (O’Connor, 2010; Yentis, 2010). While these piecemeal activities

generate more articles, these activities also circumvent any comprehensive

understanding of the explained phenomenon (O'Connor, 2010; Yentis, 2010).

Empirical Research

Academic literature, including dissertations, appears to be increasingly rich in

plagiarism research. Based on a keyword search of articles published in EBSCO from

the year 2011 to 2015, using the term “plagiarism,” EBSCO returned 2,240 published

scholarly, full text, peer-reviewed articles. An equivalent search of dissertations using

ProQuest Dissertations & Theses Global returned 5,982 documents. I limited my

literature review to published literature and excluded dissertations. I further reduced the

literature count by adding the search term “study” which then returned 176 documents.

From these documents, the relevant studies tended to focus on plagiarism from the:

• Student perspective (e.g., Chao, Wilhelm, & Neureuther, 2009; Cheema,

Mahmood, Mahmood & Shah, 2011; Hege, 2008, Siaputra, 2013)

• Faculty perspective (e.g., Allen, Ball, & Smith, 2011; Bennett, Behrendt &

Boothby, 2011; Halupa & Bolliger, 2013; Marcus & Beck 2001; Olt, 2007),

31

• Perspective of editors and reviewers (e.g., Broome, Dougherty, Freda, Kearney,

& Baggs, 2010; Elbeck, 2009; Zhang, & Jia, 2012).

Moreover, there were several studies that examined and reported levels of

plagiarism in published works (Sun, 2013; Thomas, & de Bruin, 2014). There were few

studies, which reviewed techniques for employing plagiarism detection software (Hill &

Page, 2009; Heather, 2010).

Student Perspectives

Ling (2006) interviewed 46 undergraduate students from five different language

backgrounds: Native-English-speaking (n = 11), German (n = 10), Chinese (n = 8),

Japanese (n = 9), and Korean (n = 8). He reported that:

Findings suggest that the majority of participants were not sure about whose

words and which ideas they needed to cite with acknowledgment in their writing.

Many participants who speak English as a second language (L2) also expressed

concerns about being accused of copying as innocent language learners and

some with nonwestern backgrounds also found the concept of plagiarism foreign

and unacceptable. (p. 264)

Cheema, Mahmood, Mahmood, and Shah (2011) surveyed 60 doctoral and

masters’ students about their understanding of plagiarism. While they found that

students understood the basics of plagiarism, students lacked sufficient knowledge

about specifics and the possible repercussions from being involved with plagiaristic

activities. Similarly, Orim et al. (2013) interviewed 18 students and found that they had

an inadequate understanding of plagiarism and concluded that university instruction did

not include sufficient plagiarism awareness.

32

Siaputra (2013) studied personality traits of students who plagiarize and found a

statistically significant correlation (r = .27) between levels of plagiarism and

procrastination. Hege (2008) investigated how the mood of a student could affect their

ability to identify sources from memory. She found that students in a happy mood made

more memory source errors than those in a sad mood. Thus, she concluded that

students in a happy mood would inadvertently plagiarize more often than students in a

sad mood would plagiarize.

While academic literature has identified some of the systemic issues relating to

plagiarism, there have also been corpus plagiarism studies that have tested theories of

intervention (e.g., Dee & Jacob, 2012; DeGeeter et al., 2014; Hege, 2008; Youmans,

2011). Dee and Jacob (2012) provided 28 students with instructions on “what

constitutes plagiarism and providing them with effective strategies for avoidance” (p.

423). Their results exhibited a 3.6% lower plagiarism rate when compared to a control

group who received no instruction. Chao, Wilhelm, and Neureuther (2009) studied the

effects of providing a group of students with “clear and specific instructions on reducing

plagiarism in a graded writing assignment” (p. 39). They found that students who

received instruction exhibited a 3.16% lower content similarity index than students in the

control group who did not receive the instructions. DeGeeter et al. (2014) used pre- and

post-intervention assessments to identify the effect of educational instruction on

plagiarism. They found, following the intervention, their sample of 252 students, had a

4% increase in the number of students who identified plagiarism.

Youmans (2011) conducted a similar intervention study using two groups of

students (n = 90). The instructor informed the experimental group that he would use

33

Turnitin to check for plagiarism. The control group was uninformed. The students who

were informed demonstrated no significant difference in DSI measurements (n = 44, M

= 7.59%, SD = 7.17%). In comparison, the measurements from the control group who

the instructor did not inform were (n = 46, M = 7.29%, SD = 7.10%). Youmans

concluded that informing students that their work would be checked with plagiarism

detection software was inconsequential in preventing plagiaristic student activities

(2011).

Faculty Perspectives

Halupa and Bolliger (2013) surveyed 340 faculty members (26.2% response rate)

and found that faculty perceived policies about plagiarism were not clear or understood

by faculty or students. Moreover, Bennett, Behrendt, and Boothby (2011) surveyed 159

instructors on what constitutes plagiarism and found that half of the respondents did not

see text recycling as a serious issue. Even fewer reported using software to check for

plagiarism in their work. However, concerning perceptions about faculty, Allen, Ball, and

Smith (2011) surveyed information systems researchers and faculty and found that 67%

had observed their colleagues engaged in plagiarism-of-self.

Marcus and Beck (2011) surveyed 99 speech and English faculty members for

their views on plagiarism. From 17 members who responded, they obtained 14 viable

surveys. They found that the respondents disagreed 50% of the time on what

constitutes plagiarism and were often not in accordance with their institution's policies.

Marcus and Beck proposed that faculty participate in additional training and professional

development. Olt (2007) conducted a qualitative study of responses from 28 faculties

across the U.S. on what would constitute course structure that remedies plagiaristic

34

activities by students. Resulting from her plagiarism prevention research, she proposed

a set of tasks for on-line course development and delivery, which included:

Design prevention-focused syllabi

Design plagiarism-resistant courses

Design plagiarism-resistant assignments

Ensure manageability

Model ethical behavior

Encourage interactivity

Provide feedback

Build strong relationships and trust

(Olt, 2007, p. 122)

Editor and Reviewer Perspectives

Broome, Dougherty, Freda, Kearney, and Baggs (2010) conducted a survey of

reviewers in the field of nursing. Over 1,600 respondents answered open-ended

questions involving publishing ethics. Sixteen percent of the reviewers indicated that

they were directly involved with detecting plagiaristic activities. Of the 16%, 98%

reported their findings to the editor, and 80% of those who reported were satisfied with

the outcome of their efforts.

Elbeck (2009) surveyed 26 journal editors (27% response rate) regarding

plagiarism-of-self. Elbeck found that 80% of the respondents would reject highly

plagiarized-of-self manuscripts. Additionally, 15% of the respondents would forward

evidence of severe plagiarism-of-self to the author’s department chair and college dean.

However, Elbeck found that only 12% of the respondents reported the use of plagiarism

35

detection software in the pre-screening of submitted manuscripts (2009). More recently,

Zhang and Jia (2012) surveyed 3,912 journal editors (5.6% response rate) and found

that 42% of respondents had used a plagiarism detection tool. That is a large increase

in the use of a plagiarism detections tool when compared to Elbeck’s (2009) study,

three years earlier.

Measuring Evidence of Plagiarism

Although there have been extensive descriptions of what constitutes plagiarism,

there are no finite standards on which to determine a definitive detection of plagiarism.

While most schools publicize a zero tolerance for plagiarism, professional organizations

like COPE (2014) have not been able to provide standards or definitive measurements

for evaluating plagiarism, only suggestions for handling complainant initiated plagiarism

cases.

Turnitin employed a color-coding scheme based upon the DSI (UMUC, 2016).

The color blue indicates fewer than 20 words are similar, green equates to 0% to 24%

similarity, while yellow is 25% to 49%, orange is 50% to 74%, and red is 75% to 100%

(see Table 2). Turnitin does not say whether the colors are for the DSI before any

investigation or exclusions or after a plagiarism investigator has identified and removed

all exclusions.

Occasionally, researchers have provided benchmarks based upon DSI results

that they have used in determining the extent of alleged plagiarism. For instance,

Thomas and Bruin (2014) suggested using DSI values of “1% to 9% as low; 10% to

14% as moderate, 15% to 24% as high and equal to or greater than 25% as excessive”

measures of plagiarism (p.2). The Higher Education Commission, Pakistan (n.d.)

36

published these guidelines on plagiarism levels:

If the report has [a document] similarity index <=19%, then the benefit of

the doubt may be given to the author but, in case, any single source has

similarity index >=5% without citation then it [document] needs to be

revised. (p. 3)

Zhang and Jia (2012) surveyed editors on what they perceived as troubling levels

of plagiarism and found that the

The majority of respondents indicated that if between one-quarter [25%]

and one-third [33%] of the content in the abstract, introduction or

discussion is copied without citation, the paper is likely to be rejected. (pp.

296-297)

Masic (2012) posited that if 25% or more of an article is not original, a publisher

should take remedying action. Samuelson (1994) reported that her colleagues used a

30% acceptance rule for plagiarism-of-self. While these four examples provided

guidelines, the acceptable levels of plagiarism and appropriate corrective recourses are

subjectively left up to decision makers at individual institutions involved with the

publication process (cf. Toulouse Graduate School, 2016; MSU, n. d.).

Measuring Predictors of Plagiarism

Prediction is an important part of plagiarism research. In the field of statistics,

independent variables, also called predictor variables, are used to predict the outcome

in a dependent variable, sometimes called the outcome variable (cf. Field, 2011; Howell,

2010). Predictions are not to be construed as a cause, but only as a prediction of

37

outcome. In preparation for this exploratory study, I examined several studies to review

what researchers have already investigated in the area of plagiarism prediction.

This study identified three reasons that researchers conduct predictive analytics

in empirical studies. The first reason is using prediction analytics in a “primer” document

(Petrucci, 2009, p. 193). These type of studies clarify and add to existing techniques

used in statistical prediction analyses and often include history, theoretical foundations,

example datasets, statistical software syntax, results, interpretations, and reporting

techniques. The second reason is the need to use prediction analytics to identify

potential historical or future results (cf. Fahy, 2013). The third reason is to use predictive

analytics to possibly encourage, reduce, or prevent the predicted results (cf. Youmans,

2011). The focus of this study was to promote the understanding of potential plagiarism

reduction or prevention strategies. Bertolucci (2013) calls this prescriptive analytics.

However, Siegel (2016) add another layer to the predictive model. He describes the

actionable measure of any prediction analytics and posits that practical experience,

such as business experience, is very important to determine if any prescription is

actionable.

Directly applied to the study of plagiarism, Keck, (2006) studied the quality of

paraphrasing with L1 (n = 79) and L2 (n = 74) students. Using “Near Copy, Minimal

Revision, Moderate Revision, and Substantial Revision” (p. 261) to evaluate each

student’s attempt at paraphrasing the main points in a 1000 word essay, he found that

L1 writers produced better paraphrasing results and required less rewriting efforts that

L2 students.

38

Gibelman and Gelman (2003) used a qualitative case study method and

examined what media reported about highly visible instances of academic plagiarism.

They reported that the news media tended to sensationalize accusations of plagiarism

by public figures that included prominent scholars. They concluded that student

perceptions of faculty and scholars who reap the rewards from plagiarisms predicted

student plagiarism levels.

Martin, Rao, and Sloan (2011) studied 158 students using one assignment

submitted by each of the students to Turnitin. The focus of their study was to predict

plagiarism based upon participant demographic data. Using a 3% plagiarism threshold,

they found 74% of the students plagiarized. They also used ethnic markers for

Caucasian and Asian for predicting plagiarism and found no differences that ethnicity

explained. However, acculturation, or time spent exposed to a specific culture, did show

some linkage to plagiarism. They also concluded that just knowing one’s assignment

would be examined for plagiarism was not a strong deterrent. However, they supported

the premise that having a student reviewing their COA report was an effective deterrent.

Perfect, Defeldre, Elliman and Dehon (2011) examined age and its effects on

plagiarism levels. Their study collected data on a balanced sample of young (n = 32, M

= 22.72 years, SD = 2.44) and older adults (n = 32, M = 65.22 years, SD = 4.46). While

the researchers anticipated that age predicted rates of plagiarism, they found no such

effect.

Honig and Bedi (2012) examined 279 papers by 636 authors who presented at

the International Management Division of the 2009 Academy of Management

conference. They found that 25.44 % of the corpus (71 papers) they reviewed had some

39

level of plagiarism. Furthermore, they found that 13.6 % of the corpus (38 papers) had

significant levels (5% or more of plagiarized content). Using their broad access to

participating conference authors and institutional demographics Honig and Bedi (2012)

found that newly institutionalized (periphery) countries, untenured authors, versus

tenured authors, authors degreed from non-English speaking countries and male

gender predicted higher rates of plagiarism.

Ison (2012) used Turnitin to examine 100 dissertations (average 180.4 pages)

from predominantly online institutions from the years 2009 to 2011. He reported that the

dissertations had a mean DSI of 15.1%, with a standard deviation of 12.0%, and a

range of 2% to 81%. The theme of his study was that plagiarism was prevalent at online

institutions. However, he did not investigate that theme until 2014. Ison (2014) in a

follow-up study measured plagiarism statistics from 368 dissertations. The corpus

consisted of 184 dissertations from online institutions and 184 dissertations from brick

and mortar institutions for the period 2009 to 2013. He reported that there was no

statistically significant difference between the DSI of dissertations authored by students

at online institutions and the DSI of dissertations authored by students at traditional

institutions.

Youmans (2011) studied Turnitin results for students who were aware that the

instructor was submitting their work to Turnitin and those who were unaware of that

process. He concluded that just informing students that their work would be examined

with plagiarism detection software was inconsequential in preventing plagiaristic student

activities. However, Heckler, Rice, and Bryan (2013) studied Turnitin results for students

who were required and those not required to submit their assignments through Turnitin.

40

They concluded, “The fact that there were lower rates of plagiarism when students knew

they were being monitored suggests the detection system was an effective prevention

strategy” (p. 243).

Sun (2013) examined 600 STEM and Social Science articles using Turnitin. He

identified adjusted document similarity indexes that ranged from 0% to 48% using a 30-

word exclusion parameter. However, Sun went beyond simple descriptive analysis and

explored the data using regression techniques. Sun used journal categories, disciplines,

levels of author counts (1-2 & 3 or more), and official languages for predictors in a

negative binomial regression analysis. Sun found that journal categories and disciplines

did not predict rates of plagiarism. Sun found that single or dual-authored articles had

higher similarity scores than plus three multi-authored articles. Sun also discovered that

“whether or not an author was living in a context in which English is an official language

was not significantly associated with their Turnitin score” (p. 268).

Thomas and de Bruin (2014) examined 371 articles published in South African

journals for content similarity using Turnitin. They found document similarity indexes

ranging from 1% to 91% with a mean of 17.10% (SD = 12.15%). They reported that

almost 50% of the documents had serious evidence of plagiarism (10%-14%), with

27.2% of the documents evaluated being high (15% to 24%) and 21.3% being

excessive at equal to or greater than 25%. They also examined the author count and

found a statistically significant prediction (F = 9.6, df1 = 2, df2 = 115, p = 0.0001)

between DSI for author counts less than three and those three or more. Post-hoc

testing revealed that DSIs for one (M = 15.75, SD = 6.76) and two author teams (M =

41

15.42, SD = 7.08) were statistically significantly higher than larger groups of authors (M

= 10.65, SD = 4.28).

Research Method: While various studies have examined author personality traits,

Journal affiliation, and authors’ native language to predict plagiarism outcomes, I found

no research examining the predictive effect that a study’s research method has on

plagiarism levels. Creswell (2014) believed that research methods chosen by

researchers were based upon what he refers to as a researcher’s “world view” or “a

general philosophical orientation about the world and the nature of research that a

researcher brings to the study” (p. 6). Gibelman and Gelman (2003) investigated how a

plagiarizer’s perception of academia influences plagiaristic activities. They cited the

Chronicle of Higher Education’s 2002 headline “Corruption plagues academe around

the world.” (2002, p.32). Gibelman and Gelman (2003) drew the conclusion that

Media revelations of plagiarism suggest that this occurrence has a perceived

payoff for those who risk it. Students rationalize the reasons for their misdeeds

as time-pressures, conflicting obligations, ‘everyone is doing it’ or, simply, the

information copied was there for the taking. (p. 245)

If Creswell’s “world view” is tainted by the Gibelman and Gelman plagiarism’s

“perceived payoff,” is there a possibility that a selected research method (or lack of)

could predict plagiarism? Therefore, this study used the research method (quantitative,

qualitative, and others) as a predictor variable.

Year of publication: Applying White and Arzi’s (2005) definition of longitudinal

research design, “A longitudinal study is one in which two or more measures or

observations of a comparable form are made of the same individuals or entities over a

42

period of at least one year” (p. 138). I examined the Turnitin COA report data (entity)

across the reported years of publication. While I did not find any comparable

longitudinal research, this knowledge could provide an understanding of any potential

trends in COA technology, prevention strategies, or ethical commitments, all of which

could lead to further research.

Author counts: Previous research has investigated author counts. Sun (2013)

examined author counts and found no prediction for plagiarism levels. However,

Thomas and de Bruin (2015) found a statistical difference in plagiarism rates between

articles with one or two authors as opposed to articles with three or more authors. This

study could not investigate author counts as a predictor for the dissertations as the

author count is always one author.

Word counts: Weisgrau (2011) teaches her students the importance of brevity

(concise and short) for effective writing. Moreover, Humphreys and Klein (2006) found

that word counts can predict certain outcomes. They found that in a study of social

media for online support groups, the word counts were strong predictors of hierarchical

conversations (depth). I examined word counts as a predictor of plagiarism within each

corpus.

Reference counts: Few studies explored reference counts. Bettencourt and

Houston (2001) examine the relationship between article method type and subject area

against the diversity of the references. Gipp, Meuschke, and Breitinger (2014) used

Citation-based Plagiarism Detection (CbPD) algorithms to detected plagiarism. They

explain:

CbPD algorithms consider citation proximity, overlap, order, frequency, and

43

distinctiveness to varying degrees to cover the possible citation pattern

rearrangements that can occur for different plagiarism forms. (p. 1530)

Their research demonstrated that this form of plagiarism detection was “computationally

more efficient than character-based approaches” (p. 1527). This technique lends itself

to the question can the reference count predict plagiarism?

Reviewing Plagiarism Software

While it is beyond the scope of this study to test and compare available

plagiarism software packages, it was important to at least acknowledge what is

available. I started with a research report that Technavio authored. Technavio identifies

themselves as "a leading global technology research and advisory company" (2016,

n.p.). Using market penetration formulas, Technavio compiled a list of the top eight

plagiarism detections software vendors (2016). Following are the synopses that

Technavio provided as well as any observational notes I have added.

Academicplagiarism: "Is an online software solution for individuals and

educational institutions to help them detect plagiarism. It offers editing and proofreading

services by academic experts and educators. It helps check web pages, books and

magazines, academic publications, and large databases of papers" (Technavio, 2016,

n.p.). I reviewed Academicplagiarism website (https://academicplagiarism.com/) and

found that they provided plagiarism detection web-services for all levels from the free-

limited versions to full-featured services. The posted interface resembles that of the

Turnitin COA report.

Grammarly: "Through its renowned product offering the Grammarly Editor helps

identify spelling and grammatical errors on the spot and rectifies them. With its writing

44

app, it ensures students that their content is easy to read, effective, and error free

through checks for contextual spelling mistakes, over 250 common grammar errors, and

vocabulary use. Its products are compatible with MS Office, plagiarism checker, and

native apps" (Technavio, 2016, n.p.). I tested the premium edition of Grammarly with the

plagiarism checking option. I was able to check pdf or Word documents for document

similarities. Mixed with its grammar checking capabilities, this add-in application

provided a document DSI. It also underlined individual content similarities and

provided a source link for further investigation. However, I found as one excludes false

positives, Grammarly does not update the DSI or provide a separate aDSI. Grammarly’s

plagiarism detection option was designed for the author. Teachers and Editors may find

that using Grammarly on a large scale is too demanding of their time.

PlagScan: "Is a browser based web service that checks the authenticity of

documents and detects plagiarism. It is compatible with all common file formats, such

as MS Word or PDF. The company offers PlagScan Pro with varied pricing options for

schools, universities, and faculty in the education sector" (Technavio, 2016, n.p.).

Plagscan's website https://www.plagscan.com/ identified free trial packages for private,

organization, and enterprise IT users.

BlackBoard: "Provides enterprise technology and solutions to the educational

industry globally. The company was founded in 1997 and is headquartered in

Washington, DC, US. Its offices are located in Europe, North America, Asia, and

Australia. The company offers a variety of solutions through latest technologies for

government, further education, business, higher education, and K12 schools. It serves

over 19,000 clients in more than 100 countries globally, which include 1,900

45

international institutions. It is present in the market by offering SafeAssign, an

antiplagiarism software for institutions" (Technavio, 2016, n.p.). It should be noted that

Turnitin has be integrated into Blackboard Learn for both student and instructor use.

PlagiarismDetect: "Caters to the corporate sector, education customers, and

individuals. The company offers standard and premium pricing plans to all its customers

along with free trials. It has a wide geographical presence that includes countries such

as the US, India, France, the UK, Canada, and New Zealand. The software offered can

check for documents in two languages, namely English and Spanish" (Technavio, 2016,

n.p.). The website http://plagiarismdetect.org/ provides an overview of the COA report

and instructions on how to remove false positives. The web service offers a basic COA

report and a Premium COA report that include multilevel source analysis. It appears the

charges are based on a per page submission.

EVE Plagiarism Detection System: "Provides antiplagiarism software to

education customers. The company was founded in 1997 and is headquartered in Eden

Prairie, Minnesota, US. The software, known as EVE2, assists faculty to check student

work against internet sources. It has a strong presence in the North American market"

(Technavio, 2016, n.p.). While there are many Internet references regarding EVE and

EVE2 marketed by Canexus, www.canexus.com is no longer accessible.

PlagTracker: "Caters to plagiarism detection needs of students, teachers,

publishers, and site owners. PlagTracker checks for the content that follows American

Psychological Association (APA), Modern Language Association (MLA), and Chicago

style of citations. Benefits offered to teachers include student management, custom

filter, grading system, live document view, document cross check, and tracking system"

46

(Technavio, 2016, n.p.). The website www.plagtracker.com provides account

management and COA services.

Turnitin: "Provides instructors with tools to engage students in the writing

process, provides personalized feedback, and assesses student progress over time. It

is used by more than 26 million students at 15,000 institutions in 140 countries"

(Technavio, 2016, n.p.). Given that this study uses Turnitin, the following is a further

brief review of the academic literature regarding Turnitin.

Heather (2010) quoted a JISC-PAS study the concluded that “Turnitin is the

global leader in electronic plagiarism detection, is a tried and trusted system and over

80% of UK universities have adopted it” (p. 3). Hill and Page (2009) reported that “the

results of our brief study indicate that Turnitin is the more accurate platform based on its

higher successful detection rate and the lower false detection rate” (p. 177). Turnitin

(2011) emphasizes the importance of their service as they stated:

Educators who employ the proper tools and technologies can significantly

mitigate plagiarism. For example, institutions with the widespread adoption of

Turnitin see a reduction in the unoriginal content of 30 to 35 percent in the first

year. By the fourth year, many institutions see levels of unoriginality fall by up to

70 percent. (p. 3)

However, Marshall, Taylor, Hothersall and Péérez-Martíín (2011) reaffirmed what

was described by Barrett and Malcolm (2005). Sutherland-Smith and Carr (2005), and

Jones (2008) as they concluded: “that the originality report generated by Turnitin can

only serve as a reference point and it takes human scrutiny to examine each incidence

of text-matching” (p. 267).

47

METHODOLOGY

Research Design Overview

I engineered this corpora plagiarism study for examining published HRD related

dissertations and published articles using Turnitin. This study's processes are dissimilar

from the examination of unpublished dissertations or articles. Unpublished dissertations

or articles do not carry all of the similarity 'baggage' that published works bear. That

being said, the methodologies described in this section provide basic descriptions of the

data stores (databases or spreadsheets), processes (manual or automated can be

used), and deliverables (resultant data for analysis). Figure 2 illustrates a summative

visual representation of this study including the three database objects and the four

main processes: building the corpus, collecting data on the corpus, analyzing the data,

and reporting the results. All of these processes include task objects where applicable.

I assembled and examined a balanced set of corpora from sampled dissertations

and academic publications. I extracted these documents from populations of

dissertations and academic publications representative of the broad human resource

development (HRD) field. Swanson (1995) provided a definition of HRD as “a process of

developing and unleashing human expertise through organizational development and

personnel training and development for the purpose of improving performance” (p. 208).

Moreover, Thumwimon and Takahashi (2010) outlined HRD’s three fundamental

component areas as “individual development (personal), career development

(professional), and organizational development” (p.11). Using these two HRD

definitions, I compiled a list of search filtering criteria: “human resource development”

48

or “organizational development” or “career development” or “training and development”

or “HRD.”

Figure 2. Corpus plagiarism study process chart. This figure illustrates the main processes and objects used in this study.

49

I used Turnitin to examine the individual documents for content originality or

(more commonly referred in the negative) as content similarity. Turnitin is a web-based

software service, founded in the 1990s by John Barrie, which markets itself as a

solution for identifying evidence of plagiarism (Klienfield, 2014). However, Turnitin is a

content similarity analysis (CSA) product that identifies and measures content

similarities between a submitted document and the Turnitin collection of published

documents by calculating source similarity indexes (SSI). An SSI is the identified

similarity expressed as a percentage of the submitted document that has been identified

in another document located in the Turnitin databases.

Turnitin’s quantitative factors: The DSI and supporting SSIs cannot be taken at

face value and “manual checking and human judgment are still needed” (James,

McInnis, & Devlin, 2002, p. 1). A thorough content similarity analysis requires manual

analysis of the identified content similarities by the researchers (Thomas & de Bruin,

2015). Turnitin has an exclusion process for eliminating false positives. An investigator,

upon determining the SSI is not potential plagiarism, can remove (exclude) that source

from the COA report. I examined every SSI equal to or greater than 1% for potential

exclusion. While it is not practical to examine the remaining less than 1% SSIs, this

study addressed their inclusion and exclusion using a prorating formula as described in

the study’s methodology section.

50

Turnitin provided two alternative ways of examining and excluding SSIs using

their Match Overview and All Sources options. The Match Overview lists the SSI

currently used in building the DSI. The All Sources option (see Figure 3) exhibited all

detected sources where Turnitin identified content similarities, including the individual

overlapping documents that contain the similarities.

Turnitin executes the exclusion option from both screens. However, excluding an

identified source from the originality report is a subjective researcher decision based on

the available evidence.

Population/Sample

The validity of this study was highly dependent upon the integrity of the corpus of

sampled dissertations and published articles. The dissertations came from the ProQuest

Dissertation and Thesis Publishing database. I chose ProQuest Dissertation and Thesis

Figure 3. Turnitin SSI interfaces for selecting SSI for examination and exclusion.

51

Publishing database because ProQuest headed the list of googled results from

“dissertation database” and marketed itself as a dissertation repository independent

from a university system.

As previously mentioned, I developed the search criteria using Swanson (1995)

and Thumwimon and Takahashi (2010) HRD definitions. The search terms were

“human resource development” or “organizational development” or “career

development” or “training and development” or “HRD.” By enclosing each word

combination in quotes, the results yielded a more focused corpus. For example,

“training” alone included numbers of sports preparation articles, while “training and

development” tended to focus more on HRD articles.

The search criteria targeted the dissertations’ searchable database fields (title,

keywords, and abstract) within a publishing window of 2011 to 2015. To meet the scope

of this study, I applied filters for “Full-text” and “Doctoral dissertation only.” This

ProQuest Dissertation and Thesis Publishing database query yielded a population of

910 dissertations. According to Howell (2010), a population can be any size as long as it

represents the interests of what a researcher is pursuing.

Using the Krejcie and Morgan (1970) method for determining the sample size

from a given population of 910 documents, I calculated a required minimum sample size

of 270 dissertations or about 30% of the population. I subjected each document in the

population with a random chance of inclusion so to improve the validity of the sample

and protect the anonymity of the dissertations, authors, and institutions. I used a

software application I engineered to randomly to sample a 910 population with a 270-

sample selection rate (see Figure 4). According to Moore and McCabe (1989), a

52

random sample adds to the external validity of a study when each member of the

population has an equal chance for inclusion in the sample.

The published articles came from the EBSCO collection of document databases

(EBSCO, 2016a). I examined several academic article databases, including Web of

Science. However, I chose EBSCO because of broad acceptance by colleges and

universities and its ability to export only articles where the publication text was available

for download. I used the same search criteria as the ProQuest query, “human resource

development” or “organizational development” or “career development” or “training and

development” or “HRD” with a publishing window of 2011 to 2015. I employed EBSCO

filters that demonstrated some equivalency to the ProQuest filters: Limit To options

“available text,” “references available,” and “peer reviewing”; Source Type options

“Academic Journals”; Database options “All Databases.” EBSCO specifies,

The default fields that are searched … include all authors, all subjects, all

keywords, all title information (including source title) and all abstracts. If an

abstract is not available, the first 1,500 characters of the HTML full text of the

article are searched. (2016b, n.p.)

The EBSCO query criteria yielded a population of 5,336 documents. Using the

Krejcie and Morgan (1970) calculations for determining a minimum sample size from a

given population of 5,336 produced a minimum sample size of 360 or about 7% of the

documents. The sample of articles was built by subjecting each document in the

population to a “random assignment” process (Howell, 2010, p. 3). Again, I used a

software application that random sample generator application set at a 360 count

sampling rate (see Figure 4).

53

.

Because this study considered both dissertations and published articles, I raised

the dissertations sample count beyond the minimum of 270 to 360 randomly selected

dissertations (about 40% of the 910 dissertations). Thus, the corpus now contained a

minimum of 720 documents for a “balanced design” between dissertations and

published articles (cf. Howell, 2010, p. 332), while still meeting the minimum sample

sizes (Krejcie & Morgan, 1970). Scholarly Research Tracker assigned all documents in

the corpus a unique document identification number.

To address corpus diversity concerns, I created reporting programs using the

SRT (Mayes, 2016) database to identify the frequencies of author submissions by

parsing the author lists. The corpus of 360 dissertations had 360 different authors. The

Figure 4. Random sampler software. This figure displays the interface used to manage the automated random sampling process. © Mayes 2017

54

corpus of 360 published articles had 854 authors with co-authorships ranging from 1

through 18. If the corpus was concentrated across a relatively limited number of

authors, researchers could call the validity of this study into question. Out of 852 listed

article authors, I credited only 24 with two publications while one author was credited

with three publications. These documents with duplicate authorships remained in the

study.

I then identified the academic institutions for the dissertations and academic

journals for the published articles. A compilation of institutions and publishers including

contribution frequencies were used to confirm a broad and multidisciplinary nature. The

corpus of 360 dissertations had the number of institutions totaling 156. The corpus of

360 published articles had the publisher count numbering 205. This data substantiates

the assumption of diversity and is in alignment with what Rhodes, Gelman, and

Brickman (2008) stated: “evidence obtained from more diverse sources more strongly

support a conclusion than evidence obtained from more homogeneous sources”

(p.114).

Data Collection Process 1

To control the internal validity of this study, I carefully documented and executed

the data collection process. This study collected data from two collection processes for

each of the two corpora (dissertation and published articles). The first collection process

was from the initial document retrieval and review. During the initial document review, if

I deemed the document inappropriate, I rejected the document from the sample, and

chose the next available document from the export list, thus maintaining the

randomness of selection. Reasons for rejections were limited to that the document:

55

Could not be retrieved

Was not a dissertation or a published academic article

Was incomplete or unreadable by Turnitin or me

I classified each of the 720 sampled documents as a dissertation or a published

article. I also reviewed the methodology section and looked for characteristics that

identified the document research design as an empirical study and the methodology

employed. I recorded the document research method using guidelines provided by

Creswell (2014) as listed below:

Empirical quantitative: If the document was defined as quantitative or there was

evidence of statistical results or calculations, the quantitative research method

was selected

Empirical qualitative: If the document was defined as qualitative or there was

evidence of content analysis, purposeful sampling, interviews or observations

qualitative research method was selected

Empirical mixed: If the document was defined as mixed or there was evidence of

quantitative and qualitative research, the mixed research method was selected;

or

Other: If there was no evidence of quantitative or qualitative research, then other

was selected

Moreover, I recorded year of publication, author count (author count for

dissertations was consistently one), word count, and reference count. Wilmington

(2013) provided a sample Adobe Acrobat JavaScript, which I modified to retrieve the

word counts from the collected PDF documents (see Figure 5).

However, I found one instance where this script returned an invalid word count

on a dissertation. A count of 6308 words on a 108-page dissertation seemed related to

the content using Acrobat's encryption strategy character set. I used Microsoft Word

56

word-count feature, which returned 27,361 document words. While recognizing this

anomaly must be an Adobe Acrobat program issue, the immediate concern was if

Turnitin had been able to deliver a valid COA report. An examination of the COA report

indicated that the report had produced a valid COA report.

Data Collection Process 2

The second data collection process was from the study’s corpus Turnitin

submissions. Before any Turnitin submission, I configured and verified settings to

exclude bibliography and quoted materials. These configured settings are in line with

what the Higher Education Commission, Pakistan (n.d.) promoted when instructing

many of the middle-eastern universities using Turnitin.

References/bibliography and table of contents must be removed from a

document which is submitted. If these are included the similarity index of

the document will be increased. (p. 3)

Additionally, the Turnitin settings used in this study were set to exclude content

similarities of less than ten words in sequence. This word exclusion feature is an

attempt to eliminate what Turnitin refers to as “trivial similarities” (iParadigms, 2011).

The Turnitin default setting for word exclusion is three words. Sun (2013) used a 30-

Figure 5. Adobe Acrobat JavaScript. This figure is the source code used in Adobe Acrobat to count and display the number of words and pages in the viewed document.

57

word exclusion setting. I tested all three of these word exclusion settings (3, 10, and 30)

and found the 10-word setting did eliminate additional trivial and template similarities.

However, I found that a 30-word exclusion affected the Turnitin DSI in some cases by

over 50%, bringing into question its validity. Because the corpus consisted of previously

published works, I manually removed publisher copyright pages and warnings. I found

that such common content in hundreds if not thousands of published documents

created havoc with the contents similarity verification process. One example as shown

in Figure 6 was the issue of common dissertation template text. The phrase “in partial

fulfillment of the requirements for the degree of Doctor of Philosophy” was common

across thousands of documents in the Turnitin document collection database.

Moreover, a dissertation’s table of contents and subheadings also created template

similarities. There were thousands of SSIs, which failed to provide evidence of

plagiarism. Moreover, these types of template similarities were difficult to exclude in

both dissertations and published articles.

After completing the document preparation steps, I submitted each document

using its document identifier to Turnitin for content similarity analysis to detect and mark

potential evidence of plagiarism (iParadigms, 2011). The Turnitin submission process

generated a “content originality report” that provided quantitative data and a visual

Figure 6. An example of a dissertation template.

58

highlighting of the areas within the submitted document that were similar to other

documents (see Appendix H).

Moving on to the direct human intervention part of a plagiarism analysis, I found

no fully documented steps in any of the previous studies for detecting false positives.

Moreover, Turnitin did not provide any particular documentation for evaluating content

similarities. iParadigms (2011) in its training manual states:

The decision to deem any work plagiarized must be made carefully, and only

after in-depth examination of both the submitted paper and suspect sources in

accordance with the standards of the class and institution where the paper was

submitted. (p. 50)

Thus, an important part of this study was to identify and document the process

used to evaluate the validity of the content originality report objectively. However, as the

Higher Education Commission (n. d.) stated, in the searching for evidence of plagiarism

an investigator must consider subjectively that “the benefit of the doubt may be given to

the author” (p. 3).

I identified two major processes that are required to adjust the DSI for corpus

research. I emphasize that checking student and author original work would require a

different set of processes and tasks that are outside the scope of this research.

Step 1: The DSI was the summation of the source similarly indices (SSI). The

SSI identified each specific source where Turnitin found a textual similarity. Turnitin

displayed the SSIs in two different formats: Match Overview and All Sources (see

Figure 4). The Match Overview shows the similarities that Turnitin has assigned to that

particular listed SSI. One can exclude SSI for the Match Overview by selecting the right

59

arrow for a breakdown of all related SSI that have text in common. The All Sources view

allows a researcher to view all of the sources that have similarities, even when they

contain duplicated similarities. An investigator can exclude sources from either view. I

excluded sources from the report when I suspected a false positive based upon the

preponderance of the evidence. However, often when I eliminated a source the Turnitin

statistics unexpectedly remained as they were. For instance, eliminating a source rarely

reduced the DSI by the amount attributed to the excluded SSI. That was because other

sources had the same or parts of the same similarities. This anomaly became a serious

issue when the repeated parts of a publisher’s template were common across hundreds

of articles. I used the guidelines listed below when examining SSI for a determination as

to whether to exclude them from the Turnitin reported DSI.

When an SSI source document in the Turnitin document collection database was

unavailable for inspection and the examination of the SSI content stream was

inconclusive, I excluded that SSI to prevent the false identification of content similarities

that provided evidence of plagiarism (cf. Stevens, 2009). However, listed below are the

Process 1 tasks for detecting false positives (not evidence of plagiarism) while the DSI

remained equal to or greater than 15% or when any SSI are equal to or larger than 1%.

1. Record the starting Turnitin DSI.

2. Select “All Sources” with available sub-sources.

3. Starting with the largest unexamined SSI percentage, examine the source

document or DCD text stream for identified similarities. Exclude duplicate

posting of the same document and marketing references to the title and

60

abstract (author list and date of publication are essential for a concise

evaluation.

4. If needed, examine the sub-sources for further clarification.

5. Exclude the similarity source if there is no evidence that the similarities are

plagiarism including trivial and templates similarities.

6. If the exclusion does not reduce the DSI to a level below 15% or there are

SSI 1% or greater repeat process 3 through 6.

7. When finished with the exclusion process, record the current DSI (the

beginning DSI minus exclusions) and from the Match Overview interface then

summarize the SSI greater or equal to 1% and record as the aDSI.

8. Proceed to the next Turnitin content originality report.

9. Repeat Steps 1 through 7 until corpus examination is complete.

Step 2: While Turnitin cannot identify the types of plagiarism, this study outlines a

method of identifying plagiarism-of-others as opposed to plagiarism-of-self by examining

all of the SSI equal to or greater than 5% that have not been previously excluded. I

examined SSI for the following two conditions:

Plagiarism-of-Others: If the author(s) were different and the source document

had an earlier publishing date, the SSI was identified as plagiarism-of-others.

Plagiarism-of-Self: The SSI was considered as evidence of plagiarism-of-self

If there were any author(s) common to both documents and the source document:

• Was not the author’s dissertation, thesis or other student’s papers

• Was not a non-public published conference proceedings

• Was not a grant project document or presentation

61

• Was not a duplicate web posting of the same document or abstracts

• Did not contain a note that the submission is a duplicate or variation of the

original manuscript with appropriate publisher permissions or releases

This processing of plagiarism type identification could be integrated into the

previous step when identifying exclusions. However, I chose to perform step 2 following

the completion of step1. By separating the process into two steps, I occasionally found

and corrected errors in the previous results from step 1, thus increasing the internal

validity of the study. See Figure 7 for a visual representation of the following process

descriptions in Step 1 and Step 2. The data collected from Step 1 and Step 2 were

inputted in a researcher developed SRT interface (See Figure 8).

62

Figure 7. Flow diagram for verification of Turnitin content originality report.

63

Data Export Process

After collecting all relevant data, SRT created several CSV export files adding

several derived or synthetic variables essential to the analysis. Because of the wide

range of the word counts, it became apparent the 10% of a 100,000-worded document

was not the same as 10% of a 10,000-worded document. While the scale of choice for

Turnitin is solely based upon percentages, this study also included the word counts

associated with these percentages. During the SRT data export process, the SRT

application calculated the DSW and aDSW.

Figure 8. SRT Turnitin COA report data collection interface.

64

While Turnitin does not provide a method for summarizing the content similarities

that were identified as less than 1%, SRT estimated their contribution to the aDSI by

subtracting the sum of the remaining SSI equal to or greater than 1% from the current

document similarity index (the starting DSI less any exclusions). Moreover, based upon

the percentage of posted exclusions, SRT interpolated how much of the SSIs less than

1% would be legitimate similarities and a part of the aDSI. Below is the formula I used in

SRT on each document (d) with an example that illustrates how it was applied:

𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑑𝑑 = �𝑎𝑎𝑎𝑎𝑎𝑎(=>1)𝑑𝑑 + � 𝑒𝑒𝑎𝑎𝑎𝑎𝑎𝑎 𝑑𝑑𝑎𝑎𝑎𝑎𝑎𝑎𝑑𝑑

× �𝑎𝑎𝑎𝑎𝑎𝑎(<1)𝑑𝑑�

For example, document (d) has a Turnitin reported DSId value of 45%. After the

investigator had excluded several SSId believed not to be evidence of plagiarism, the

DSId was reduced to a new value eDSId of 10%. That produced a ratio of 10/45 or 22%

of the summation of all of the Turnitin reported SSI as evidence of plagiarism. However,

if the investigator summarizes the remaining SSI(=>1)d equal to or larger than 1% and

finds they add up to 6%, the remaining 4% are the sum of the SSI(<1)d smaller than 1%.

The ratio of 10/45 or 22% was applied to the 4% producing .88% to be then added to

the 6%. This document then had an estimated 6.88% or rounded up to 7% of its content

as similarities that were evidence of plagiarism. I used a spreadsheet formula to double-

check the accuracy of the automated SRT formula (see Figure 9).

I used the SRT software (Mayes, 2016) to export the data collected into CSV files

for importation into IBM’s Statistical Package for the Social Sciences (SPSS) Version

20. During the export process, I used an SRT automated process that used the aDSI

values to assign DSL membership values. I addressed each research question using

65

SPSS for the analysis and R-Studio for graph and charts. I saved both the data files and

the syntax files for further reference.

Data Collection Summary

In review, the Turnitin DSI was the most important or visible variable provided by

Turnitin. However, the primary quantitative dependent variable in this study was the

adjusted document similarity index (aDSI). The aDSI was the sum of the remaining non-

excluded SSI equal to or greater than 1% plus a proportional amount of the remaining

SSI that are less than 1% as previously described.

I used the aDSI to generate the categorical document similarity level (DSL)

values. The DSL was categorically modeled using three membership levels of aDSI

based upon previous research and instructional documents (see Table 2.). Using

ordinal categories, I denoted the first membership level as acceptably low or moderate

aDSI with a range 0% to 14%. The second membership level was denoted as high

using aDSI with a range of 15% to 24%. The last membership level was denoted as

excessive using aDSI with a range equal to or greater than 25%.

Figure 9. Spreadsheet formulas for double-checking SRT-DSI final adjustments.

66

Data Analysis

To reiterate, this study analyzed data from corpora including a dissertation

corpus and a published article corpus. While the dissertation corpus and the published

article corpus were from the HRD field, they had many distinct differences. First,

dissertations always had one author, while this sample of published articles had up to

eighteen authors. While dissertations primarily employed empirical research methods,

published articles included editorials, theory development, and other areas of

professional interests. Moreover, it was common for dissertations to have over 100,000

words while published articles rarely exceeded 10,000 words. With these large

differences, statistical significance in Corpus comparisons could be difficult to interpret.











I addressed both RQ1.1 and RQ1.2 through SPSS and R descriptive statistics

that included distributions, measures of central tendencies, frequencies, and Pearson r

correlational statistics that identified relationships between the variables: author, words,

67

and reference counts (Bedeian, 2014). Next, I examined the data for outlying conditions

and abnormalities that may indicate inappropriate sampling, or a problem as simple as a

mistake in entering or coding data, (Mullen, Milne & Doney 1995). However, as Grace-

Martin (2016) states: “It is NOT acceptable to drop an observation just because it is an

outlier. They can be legitimate observations and are sometimes the most interesting

ones” (n.p.). Retaining outliers is certainly prudent in plagiarism studies.

For the continuous variables DSI, DSW, aDSI, and aDSW, the mean, standard

deviations and ranges, I used the descriptive statistics features of SPSS for the two

corpora of dissertations and published articles and various subpopulations. For the

continuous variables: author, word, and reference counts, the means and standard

deviations I again used the descriptive statistics features. For the polychotomous

variable research methods (quantitative, qualitative, and other) and for the DSL

dependent variable I conducted frequency and subpopulation descriptive statistics.







Both RQ2.1 and RQ2.2 were important research questions. Researchers have a

need to know if the Turnitin provided DSI values are credible or are they significantly

different from researcher generated aDSI values. Researchers can expend a significant

68

amount of resources to calculate aDSI values. If there is a statistically insignificant

difference, that investigation process may be unimportant.

I used a Wilcoxon Signed-rank statistic, to analyze the statistical difference or

lack of, between the two dependent variables. Researchers often use the paired t-test

statistic for this type of statistical analysis, but the paired t-test has the assumption of

normality on the difference between the two dependent variables. However, our pilot

study indicated that the data difference histogram was bimodal and heavily skewed. We

confirmed this abnormality condition using the Shapiro-Wilk test for normality. Both the

dissertations and published articles have fewer than 2000 elements, so I chose the

Shapiro-Wilk, over the Kolmogorov-Smirnov test for normality. For either test, if the

results showed a statistical significance (p < .05) the data would not meet the

assumption of normality on the difference between the two dependent variables.

The Wilcoxon Signed-rank sum statistic with α = .05 was substituted for a t-test

analysis. The Wilcoxon signed-rank statistic works with the variables distribution

elements in order of size and then determines the difference between the two ranked

distributions. Another difference is that the Wilcoxon signed-rank statistic is dependent

upon the median value of the ranked distribution. Box plots are normally used with

Wilcoxon Signed-rank statistic (Massart, Smeyers-Verbeke, Caprona & Schlesierb,

2005). According to Zimmerman (1996):

The Wilcoxon test is more powerful for various non-normal distributions with

excess skewness and kurtosis, including mixed-normal, exponential, Cauchy,

and Laplace distributions (Hodges & Lehmann, 1956; Randles & Wolfe, 1979).

The comparison does not imply that violation of the normality assumption has no

69

influence on the Wilcoxon test. On the contrary, the power of the test to detect

alternatives declines, despite maintenance of the significance level. But the

power of the t-test declines even more, so the nonparametric method acquires an

advantage. (p.29)

The Wilcoxon signed-rank sum effect size (r) is widely used and understood

(Rosnow & Rosenthal, 2005). The Wilcoxon signed-rank sum effect size (r) is based

upon the calculated Z score divided by the square root of the number of data elements

(Field, 2011, pp.26 & 558). The key to the symbols: 𝑍𝑍 = Z score, 𝑋𝑋 = mean of difference

between the conditions, s = standard deviation, N = number of cases.

𝑧𝑧 = 𝑋𝑋−𝑋𝑋𝑠𝑠

𝑟𝑟 = 𝑍𝑍√𝑁𝑁

As with the t-test, the Wilcoxon Signed-rank sum effect size (r) is interpreted

much like a Cohen’s d. Olive and Franco (2008), as cited in Cohen (1988), suggested

using a Cohen’s d = 0.2 is a small effect size, d = 0.5 is a medium effect size and d =

0.8 is a large effect size. Rosnow and Rosenthal further state a Wilcoxon Signed-rank

effect size estimate r reported value of 0.5 and above is large (2005).


reference count predict membership in low, high, or excessive levels of the plagiarism





70

Both RQ3.1 and RQ3.2 employed the ordinal DSL as the dependent or outcome

variable. However, while RQ3.1 employed year of publication, research method, word

count, and reference count as four independent or predictor variables, RQ3.2 also

included author count. I addressed the analysis of these variables with a multinomial

logistic regression (MLR) analysis, modeling the three levels of content similarity in what

was referred to as the dependent, respondent, or outcome variable. Multinomial logistic

regression (MLR) is a not a member of the generalized linear models (GLM), but rather

uses a logarithmic transformation to change categorical data in a linear way (Bham,

Javvadi & Manepalli, 2012; Petrucci, 2009; Berry & Feldman, 1985). However, MLR’s

strength is that both continuous and categorical independent variables can be used to

predict membership in a categorical dependent variable (O’Connell & Rivet Amico,

2010). The multinomial dependent variable was chosen over a binomial dependent

variable because it was based upon the three categories of similarity levels as defined

in the merger of several standards from recommendations such as Thomas and Bruins

(2014), the Higher Education Commission, Pakistan, (n.d.) and others. The standards

used in the analysis for this study were low (0% to 14%), high (15% to 24%) and

excessive (equal to or greater than 25%).

I did consider predictive discriminant analysis (PDA), but because of the potential

mixture of continuous, dichotomous, and polychotomous independent variables, MLR

was the final choice (Petrucci, 2009). Moreover, MLR in SPSS is easier to use and

more robust than PDA (Burns & Burns, 2009) and uses odds ratios for measuring

predictability. Odds ratios for each independent variable look at the sample size and the

total number of the independent variable cases that succeeded in changing the

71

dependent variable (IDRE-UCLA, 2017A). Odds ratios are easier to understand and to

make comparisons. For example, MLR is often used to predict buying decisions such as

“Do women prefer SUV vehicles over sedans and trucks?” Using a multinomial logistic

regression with the dependent variable being the purchase of an SUV or sedan or truck

and the independent variable would be vehicle purchases decision maker being a

male=0 and female=1. Based on the historical data, a multinomial logistic regression

could show that if a female made a purchase, she would be 6.5 times more likely to buy

an SUV over a sedan and 29.6 times more likely to buy an SUV over a pickup.

The collected data was examined for statistical assumptions as required for MLR

model analysis. First, each variable must have a single value per case. By design, I did

not have duplicate cases. Also, the MLR model assumes that there are no independent

variables, which can perfectly predict the dependent variable. While several of

independent variables were categorical, there were continuous variables. Thus I did

have a reason to test the assumption for linearly related to the logit (Field, 2011).

Moreover, over-dispersion is a serious issue if the assumption of independence of

errors is not met (Field, 2011). Field (2011) stated that while independent variables do

not have to be statistically independent of each other, there is an assumption that

multicollinearity is relatively low in both linear and logistic regression. High levels of

predictor multicollinearity can affect the results. For example, if word count and

reference count are highly correlated, that may affect the logistical regression statistics.

I tested word counts and reference counts for multicollinearity. There were no

multicollinearity issues.

https://en.wikipedia.org/wiki/Statistically_independent

72

Sample size guidelines for multinomial logistic regression indicate a minimum of

10 cases per independent variable (Schwab, 2002). That guideline was exceeded with

the 360-document sample. Field (2011) and Anderson (1984) warn that empty cells or

zero cases in a subpopulation are an issue with logistic regression. I found that several

research methods did not fully populate the subpopulation cells. I combined them into

the "Other" class within the research method variable. That eliminated the empty cells

without reducing the sample size.

I conducted the multinomial logistic regression analyses for each level in the

dependent variable categories (low, high, and excessive) as the specified reference

base. Moreover, I conducted the MLR analyses for all possible subsets of the

independent variables. The resulting Nagelkerke's Pseudo R2 values explained the

importance of the predictors in terms that closely behave like a linear model (Allison,

2013). Bewick, Cheek, and Ball (2005) explained that Nagelkerke's Pseudo R2

demonstrates “how useful the explanatory variables are in predicting the response

variable and can be referred to as measures of effect size” (p. 116). These subset

analyses provided the data needed to determine the best combination of the

independent variables in the final MLR model.

73

RESULTS

This chapter contains the data analysis results from the dissertation corpus,

followed by the results from the published article corpus. I included the research

questions followed by the respective results. Various tables and brief explanations

present the results for reader review.

The following is a quick review of the pertinent acronyms, starting with the

document similarity index (DSI). The DSI is a summation of the applied source similarity

indices (SSI) as reported by Turnitin before any researcher exclusions or adjustments.

The adjusted document similarity index (aDSI) is the DSI less any researcher

exclusions or adjustments. The substantive SSIs are those SSIs equal to or larger than

5%. The document similarity level (DSL) is the dependent variable derived from aDSI

range values. I derived the document similarity words (DSW) and adjusted document

similarity words (aDSW) from the individual document DSI and aDSI (both percentages)

as applied to the document word count variable.

Dissertation Descriptive Statistics Results






I conducted a descriptive statistical analysis on the corpus of 360 HRD related

dissertations (see Tables 4, 5, and 6). Table 4 exhibits the means statistics for relevant

variables across the corpus of dissertations. Most important are the dissertation aDSI

74

values of M = .09 (SD = .06). Only nine of the 360 dissertations had any non-excluded

substantive SSIs. Thus, the SSI means were reported at extremely low levels. Most

noticeable are the large kurtosis values for many of the variables.

Removal of outliers is a common way to add normality to the sample, but as

Grace-Martin (2016) stated, “It is NOT acceptable to drop an observation just because it

is an outlier. They can be legitimate observations and are sometimes the most

interesting ones” (n.p.).

Noticeably absent in Table 4 and 5, is any reference to substantive plagiarism-of-

self SSI data in the dissertations. I did not find any substantive plagiarism-of-self SSI

data in the review of the SSI. However, I did find one 4% plagiarism-of-self SSI value. I

Table 4

Corpus Wide Descriptive Statistics for Dissertations (n=360)

Variable Min Max M SD Skewness S.E. Kurtosis S.E.

Year of Publication 2011 2015 2013 1.36 .20 .13 -1.19 .26

Author Counta 1 1 1 .00 .00 .00 .00

Word Count 13160 155389 48877 22833 1.34 .13 2.17 .26

Reference Count 1 602 140 76 1.69 .13 5.19 .26

DSI .01 .98 .27 .28 1.45 .13 .60 .26

DSWb 822 141404 13434 19018 3.05 .13 11.18 .26

aDSI .00 .40 .09 .06 1.29 .13 3.13 .26

aDSWb 0 19990 3828 2484 1.54 .13 3.76 .26

SSI Other Countc 0 3 .04 .31 8.11 .13 69.59 .26

SSI Other Similarity Mc .00 .16 .03 .02 7.43 .13 56.17 .26

SSI Other Wordbc 0 4980 73 496 7.43 .13 58.05 .26 aAuthor Count for dissertations was always 1. bDSW, aDSW and SSI word counts are whole word similarities . cSubstantive SSI or SSI equal to 5% or larger.

75

mention this because it is important to demonstrate that dissertations are not immune to

plagiarism-of-self issues.

A basic understanding of the relationships between the variable data collected in

a study is important and a precursor to any predictive statistics (Field, 2011). Normally,

a Pearson's r correlation coefficient is the statistic used to identify the relationship

between two variables. However, a Pearson's r correlation coefficient statistic has an

assumption of data normality (Field, 2011). I tested all of the variables using a Shapiro-

Wilk test for normality. All of the variables violated the assumption of normality (p < .05).

There are two camps on the importance of the assumption of normality for a Pearson’s r

statistic. Nefzger and Drasgow (1957) posited, "Tests of significance of r do not in

practice require normally distributed variates" (p. 623). However, Field (2011) stated

that the Spearman's rho Correlational Coefficient Statistic is a non-parametric statistic

that is used for better results when the data violate the assumption of normality. Table 5

identifies the relationships between the variables using the Spearman's rho correlational

coefficient statistic. The results show a very strong correlation between word counts and

reference counts and very high correlations between the various similarities

percentages and word counts, including a 1 to 1 correlation with substantive SSI of

others between the percentages and words.

76

Table 5

Corpus Wide Spearman's rho Correlational Coefficient Statistic for Dissertations (n=360)

Variable 1 2 3 4 5 6 7 8 9 10 11

1 Year of Publication 1.00

2 Author Countb - 1.00

3 Word Count <.01 - 1.00

4 Reference Count <.01 - .49a 1.00

5 DSI -.06 - -.13a .13a 1.00

6 DSW -.07 - .32a .34a .87a 1.00

7 aDSI <.01 - -.28a .04 .14a .04 1.00

8 aDSW <.01 - .27a .30a .04 .22a .81a 1.00

9 SSI Other Count -.03 - -.17a -.07 .18a .13a .25a .16a 1.00

10 SSI Other Similarity M -.03 - -.17a -.07 .18a .13a .25a .16a 1.00a 1.00

11 SSI Other Word Count -.04 - -.17a -.07 .18a .13a .25a .16a 1.00a 1.00a 1.00 aCorrelation is significant at the p < 0.05 level (2-tailed)

77

Continuing forward to Table 6, I provided additional descriptive statistics for the

various subpopulations in several of the continuous and categorical variables. Table 6

provides variable and subsample frequencies including aDSI and aDSW statistics by

year, research method, author count (one author for dissertations), and DSLs.

For additional insight, I calculated the aDSI and the aDSW within each

subpopulation. While this information did not answer any particular research question, it

provided the ability to observe the diversity of group memberships and congruence in

calculated values. Moreover, the data visually demonstrated potential prediction issues.

Table 6

Descriptive Statistics for Dissertations Subpopulation (n = 360)

Variable Sub-Populations

Freq. Percent aDSI (range) SD aDSW SD

Year of Publication 360 100.0 .09 .09 3828 2484

2011 90 25.0 .09 .07 4171 3638

2012 78 21.7 .08 .05 3904 2659

2013 76 21.1 .07 .05 3446 2538

2014 71 19.7 .09 .06 3817 2729

2015 45 12.5 .09 .05 3816 2272

Research Method 360 100.0 .09 .09 3828 2484

Quantitative 138 38.3 .11 .06 4058 2329

Qualitative 160 44.4 .07 .05 3697 3239

Other 62 17.2 .08 .06 3685 3011

Author Counta 360 100.0 .09 .09 3828 2484

1 360 100.0 .09 .09 3828 2484

Document Similarity Levels 360 100.0 .09 .09 3828 2484

Low 317 88.1 .07 (.00-.14) .04 3277 2339

High 35 9.7 .18 (.15-.24) .03 7350 2287

Excessive 8 2.2 .30 (.26-.40) .05 10486 5119

aAuthor Count for dissertations was always 1.

78

Regarding dissertations, I noticed three issues: a) The quantitative research method

documents returned the highest aDSI, b) there were no plagiarism-of-self substantive

SSI, and c) DSL membership was highly skewed to the low level. The research method

identified as 'Other' was a catchall that included the Ph.D. candidates' descriptions of

their dissertations as a mixed method, literature review, an action project, an

organization review, and an untested survey design. See Appendix A for a further

dissertation frequency breakdown of the research methods categorical variable within

each DSL, word count groupings, and reference count groupings.

Dissertation Differential Statistics Results

An important part of this study involved the Turnitin DSI and the importance of

making adjustments to or exclusions from the DSI. This lengthy process included many

manual validation procedures (see Methodology section). RQ2.1 and RQ2.2 provided

the justifications that these processes were required. It is important to know whether the

differences between the DSI and the aDSI are statistically and practically significant for

both the dissertation and the published article corpora.




Researchers often use the paired t-test statistic to test for differences between

pairs of data. In this study, the intervention was the removal of SSI false positives.

However, a paired t-test assumes data normality of the difference between the DSI and

aDSI variables.

79

Figure 10 employed histograms, which provided the visual differences between

the dissertation values of the Turnitin reported DSI and my reported aDSI. The left and

right histograms were heavily skewed to the left. The left DSI histogram exhibited minor

bimodal plotting. These histograms indicated the presence of non-normality

distributions.

Moreover, the observed statistical difference between the DSI and the aDSI

reported a skewness statistic of 1.59 (SE = .129), Z = 12.32 and a kurtosis statistic of

.852 (SE = .256), Z = 3.33. Figure 11 in combination with these statistics demonstrate

that the difference potentially violates the t-test assumption of normality (Cramer, 1998;

Cramer & Howitt, 2004).

Figure 10. Dissertation DSI & aDSI frequencies before and after the adjustments.

The black bars represent the Document Similarity Levels separators for Low, High, and Excessive.

80

To confirm the violation of the assumption of normality I conducted a Shapiro-

Wilk Test for Normality on the difference to meet or reject the assumption of normality

(Shapiro & Wilk, 1965; Field, 2011). I chose the Shapiro-Wilk Test for Normality over the

Kolmogorov-Smirnov Test for Normality because the Shapiro-Wilk Test for Normality is

applicable for matched pairs. The name of this test is somewhat misleading in that it

tests for the presence of data abnormality. The results reported a statistically

significance statistic of .629 (df = 360, p < .05). Thus, the data tested did meet the test

for the presence of data abnormality, which did not meet the assumption of normality

that is preferred for a t-test.

Figure 11. Histogram exhibiting difference between dissertation paired DSI & aDSI.

81

Based upon the non-normality of the data, I conducted a Wilcoxon Signed-rank

statistic to identify the statistical significance of the difference (Zimmerman, 1996; Field,

2011). As previously discussed in the Methodology section, the Wilcoxon Signed-rank

non-parametric statistic ranks the data sets (large to small) and then compares and

measures the difference much like a t-test. According to Zimmerman (1996):

The Wilcoxon test is more powerful for various non-normal distributions with

excess skewness and kurtosis, including mixed-normal, exponential, Cauchy,

and Laplace distributions (Hodges & Lehmann, 1956; Randles & Wolfe, 1979).

The comparison does not imply that violation of the normality assumption has no

influence on the Wilcoxon test. On the contrary, the power of the test to detect

alternatives declines, despite maintenance of the significance level. But the

power of the 't-test' declines even more, so the nonparametric method acquires

an advantage. (P. 29)

The Wilcoxon Signed-rank test confirmed the significance of the difference

between the DSI and the aDSI Z = -14.214 (p < .05). The Wilcoxon Signed-rank effect

size estimate r value of .52 (p < .05) provided strong support that these adjustments

were statistically and practically significant (Field, 2011; Rosenthal, 1991, p.19).

Rosnow and Rosenthal stated a Wilcoxon Signed-rank effect size estimate r reported

value of 0.50 and above is large (2005).

82

Dissertation Predictive Statistics Results


reference count predict membership in low, high, or excessive levels of the plagiarism


I conducted a multinomial logistic regression (MLR) analysis to predict

dissertation membership outcomes in the DSL. The goal of this MLR analysis was to

determine what independent variables or a combination of, influenced the probability of

a document belonging to one of the three DSLs. As previously discussed, the DSL is a

multinomial dependent variable based upon three categories using the aDSI with the

following membership criteria:

1. Low: Less or equal to 14%

2. High: Between 15% and 24%

3. Excessive: Equal or greater than 25%

Overall, within the corpus of 360 dissertations, SRT identified 317 cases or

88.1% in the low plagiarism range, 35 cases or 9.7% in the high plagiarism range, and 8

cases or 2.2 % in the excessive plagiarism range (See Figure 12). However, this

skewed DSL membership also caused subpopulation membership issues. The

document research method “Mixed” had only two cases in the high DSL and one case

in the excessive DSL. The document research method “Other” had one case in the high

DSL and zero cases in the excessive DSL (see Appendix A). Field (2011) and Anderson

(1984) warned that empty or minimally populated cells are an issue with logistic

regression. After careful consideration, I moved the research method "mixed" cases into

83

the research method "other" cases to eliminate the empty cells, thereby improving the

model fit.

An important part of the MLR statistic process is configuring the best model. I

used the Nagelkerke's Pseudo R2 to design the strongest model. The Nagelkerke's

Pseudo R2 is a model based pseudo effect size statistic used in MLR to explain the

importance of the predictors in terms that closely behave like a linear model (Allison,

2013). Bewick, Cheek, and Ball (2005) explained that Nagelkerke's Pseudo R2

demonstrates, “how useful the explanatory variables are in predicting the response

variable and can be referred to as measures of effect size” (p. 116). To engineer the

best model, I used a Word VBA script that listed out the all-possible-subsets (see Figure

Figure 12. Dissertation membership in DSL groups.

84

13). I then conducted an MLR on each of the possible variable combinations using

SPSS.

While the Nagelkerke's Pseudo R2 is valuable in MLR for determining the best

MLR model, it should be noted the Nagelkerke's Pseudo R2 "cannot be interpreted

independently or compared across different datasets" (IDRE-UCLA, 2017B, p.7). With

the available predictors arranged in all of the possible combinations, the highest

Nagelkerke's Pseudo R2 of .169 was achieved using a model of Year of Pub, Research

Method, Word Count and Reference Count (see Table 7).

Figure 13. MS Word VBA script for building all possible string subsets. This script creates a new MS Word document with the full listing of the subsets.

85

Table 7

Modeling MLR Analysis of Sampled Dissertations

Model

Classification Percentages Effect size (Pseudo R2)

Low High Exces-sive

Total Cox & Snell

Nagelkerke Mcfadden

Research Method Year of Pub Word Count Reference Count

99.7 0.3 0.0 87.8 .097 .169 .120

Research Method Word Count Reference Count

99.7 0.3 0.0 87.8 .091 .159 .112

Year of Pub Word Count Reference Count

99.7 0.3 0.0 88.1 .082 .143 .101

Word Count Reference Count

100 0.0 0.0 88.1 .078 .136 .095

Research Method Year of Pub Reference Count

100 0.0 0.0 88.1 .064 .111 .078

Research Method Year of Pub Word Count

100 0.0 0.0 88.1 .056 .097 .068

Research Method Reference Count

100 0.0 0.0 88.1 .056 .098 .068

Research Method Word Count

100 0.0 0.0 88.1 .048 .084 .058

Research Method Year of Pub

100 0.0 0.0 88.1 .050 .088 .061

Research Method 100 0.0 0.0 88.1 .042 .073 .051

Year of Pub Word Count

100 0.0 0.0 88.1 .027 .048 .033

Word Count 100 0.0 0.0 88.1 .022 .039 .027

Year of Pub Reference Count

100 0.0 0.0 88.1 .015 .025 .017

Reference Count 100 0.0 0.0 88.1 .010 .018 .012

Year of Pub 100 0.0 0.0 88.1 .005 .008 .006

86

Interpreting an MLR analysis requires an understanding of odds ratios. Odds

ratios are not the same as a predictive value (Grace-Martin, 2017). A continuous

variable odds ratio reflects the change in its odds ratio per one unit increase change in

the continuous variable. In this study one unit of change for words is 100 words and one

unit of change for references is 20 references. For example, using these values for the

reference variable, one unit of reference increase change (20 references) has an odds

ratio increase of 1.599 that it will belong to the high DSL over the low DSL as long as all

other variables remain constant. The low DSL is the reference category.

The odds ratios for a categorical variable like the research method variable also

assume all other variables remain constant. If one ignores the statistically insignificant

value (p = .762) for the categorical quantitative research method in the excessive DSL,

the odds ratio of 1.415 to 1 can be interpreted as a document which used quantitative

research method was 1.415 times likely to belong to the excessive DSL over the low

DSL as compared to a document one using the other research method (other is the

categorical comparison base). Likewise, if one ignores the statistically insignificant value

(p = .448) for the qualitative research method the odds ratio .332 to 1 interprets as the

qualitative research method is 68% (1 - .332) less likely to remain in the excessive DSL

over the low DSL as compared to the other research method category.

In summary, Table 8 exhibits statistically significant odds ratios for word counts

and reference counts at both the high and excessive levels. The model's overall

practical effect size using the Nagelkerke Pseudo R2 was .169. The practical effect size

is small.

87

Figure 14 provides visual representations in the form of scatter plots for both the

continuous independent variables word count and reference counts. What is noticeable

is the heteroscedasticity in the word count scatterplot and the opposing directions of the

regression lines between the word and reference count independent variables.

Table 8

MLR Analysis of Sampled Dissertations (Best Model) n=360

Dependent Variable Levels Independent Variables

B Std Error Wald df Sig Exp(B)a Odds Ratio

95% CI

LB UB

High Document Similarity Level Publication Year Word Count (x100) Reference Count (x20) Qualitative Method Quantitative Method Other Methodb

445.994 -.223 -.380 .470 .970 .162 .0

277.477 .138 .145 .152 .663 .689

2.583 2.611 6.851 9.594 2.141 .055

1 1 1 1 1 1 0

.108

.106

.009

.002

.143

.814

----- .800 .684 1.599 2.639 1.175

----- .611 .515

1 .188 .719 .305

----- 1.049 .909

2.153 9.680 4.535

Excessive Document Similarity Level Publication Year Word Count (x100) Reference Count (x20) Qualitative Method Quantitative Method Other Methodb

388.141 -.194 -.881 .877 .347

-1.012 .0

553.192 .275 .380 .310

1.145 1.451

.492

.499 5.362 8.014 .092 .576

1 1 1 1 1 1 0

.483

.480

.021

.005

.762

.448

------ .824 .414 2.403 1.415 .322

----- .481 .197

1.310 .150 .019

------ 1.411 .873 4.409

13.353 .000 .649

aConfidence Intervals for Exp at 95% bThis parameter is set to zero because it is the Research Method parameter reference category.

Note: Comparison Category is Low Document Similarity Level. Bold signifies statistically significant.

88

Published Article Descriptive Statistics Results






I also conducted a descriptive statistical analysis on the corpus of 360 HRD

related published articles (see Tables 9, 10, and 11). Table 9 exhibits the mean

statistics for relevant variables across the corpus of published articles. For instance,

published articles had a wide range of document characteristics such as a one-page

editorial at 699 words with one reference and the largest 53-page article at 24,463

words. Table 10 identifies potential relationships between the variables. Table 11

Figure 14. Dissertation scatter plots for word count and reference count variables. The green and orange lines divide the three DSL levels and the red line is the regression line.

89

provides frequencies and group aDSI and aDSW statistics by year, research method,

author counts, and DSLs.

Regarding Table 9, one must note that the descriptive statistics were based upon

the sample of 360 published articles. That was important for the substantive SSI. There

were only 81 out of 360 published articles with any substantive SSI. Thus, the

substantive SSI means were diluted down to extremely low levels. Table 10 provided

additional statistics on substantive SSI. Most important are the aDSI M = .11 (SD = .10)

and the substantive SSI other similarity of M = .01 (SD = .04) and substantive SSI self-

similarity M = .04 (SD = .12). As with the dissertations, the kurtosis statistics for the

published articles were very high across most of the variables. As previously discussed,

the removal of outliers is a common way to add normality to the sample, but as Grace-

Martin (2016) stated: “It is NOT acceptable to drop an observation just because it is an

outlier. They can be legitimate observations and are sometimes the most interesting

ones” (n.p.).

90

Table 9

Corpus Wide Descriptive Statistics for Published Articles (n=360)

Variable Min Max M SD Skewness S.E. Kurtosis S.E.

Year of Publication 2011 2015 2013 1.27 .24 .13 -1.21 .26

Author Count 1 18 2.49 1.86 3.19 .13 18.18 .26

Word Count 699 24463 8552 4186 .95 .13 1.37 .26

Reference Count 1 257 46 34 2.05 .13 7.88 .26

DSI .02 1.00 .67 .35 -.83 .13 -1.07 .26

DSWa 82 23484 5877 4516 .95 .13 1.04 .26

aDSI .00 .84 .11 .10 2.67 .13 10.40 .26

aDSWa 0 7823 900 956 2.83 .13 11.97 .26

SSI Other Countb 0 4 .12 .52 5.46 .13 32.58 .26

SSI Other Similarity Mb .00 .30 .01 .04 5.31 .13 29.41 .26

SSI Other Word Countab 0 4889 65 355 9.42 .13 110.17 .26

SSI Self Countb 0 6 .35 .91 3.36 .13 12.38 .26

SSI Self Similarity Mb .00 .96 .04 .12 4.22 .13 22.27 .26

SSI Self Word Countab 0 6692 328 977 3.93 .13 16.98 .26

aDSW, aDSW and SSI word counts are whole word similarities.

bSubstantive SSI or SSI equal to 5% or larger.

A basic understanding of the relationships between the variable data collected in

a study is important (Field, 2011). Normally, a Pearson's r correlation coefficient is the

statistic used to identify the relationship between two variables. However, Pearson's r

correlation coefficient has an assumption of data normality. I tested all of the variables

using a Shapiro-Wilk test for normality. All of the variables violated the assumption of

normality (p < .05). There are two camps on the importance of the assumption of

normality for a Pearson’s r statistic. Nefzger and Drasgow (1957) posited, "Tests of

significance of r do not in practice require normally distributed variates" (p. 623).

However, Field (2011) stated that the Spearman's rho correlational coefficient statistic is

91

a non-parametric that researchers use when the data violate the assumption of

normality. Table 10 identifies the relationships between the variables using the

Spearman's rho correlational coefficient statistic.

Moving on to Table 11, the analysis provided additional descriptive statistics for

various membership levels in several of the continuous and categorical variables. I

provided frequency counts as well as the mean statistics for the aDSI and the aDSW

within each group and subpopulation for additional insight. See Appendix B for a further

frequency breakdown of the research methods categorical variable within each DSL,

word count groupings, and reference count groupings.

92

Table 10

Corpus Wide Spearman's rho Correlational Coefficient Statistic for Published Articles (n=360)

Variable 1 2 3 4 5 6 7 8 9 10 11 12 13 14

1 Year of Publication 1.00

2 Author Count .02 1.00

3 Word Count -.02 .12 a 1.00

4 Reference Count .01 .17 a .76a 1.00

5 DSI -.17a .08 .04 .10 1.00

6 DSW -.13a .12a .62a .50a .68a 1.00

7 aDSI -.08 .08 -.04 .12a .08 .03 1.00

8 aDSW -.08 .15a .44a .47a .10 .33a .84a 1.00

9 SSI Other Count .08 -.03 -.09a -.03 <.01 -.05 .35a .25a 1.00

10 SSI Other Similarity M .07 -.03 -.09 -.02 <.01 -.05 .35a .25a 1.00a 1.00

11 SSI Other Word Count -.07 -.03 -.08 -.02 <.01 -.05 .35a .25a 1.00a 1.00a 1.00

12 SSI Self Count -.09 -.01 -.04 .08 .03 .04 .56a .52a .16a .16a .16a 1.00

13 SSI Self Similarity M -.09 -.01 -.04 .09 .03 .04 .57a .52a .17a .17a .17a 1.00a 1.00

14 SSI Self Word Count -.09 -.01 -.06 .11a .03 .05 .57a .52a .16a .16a .16a 1.00a 1.00a 1.00

aCorrelation is significant at the p < 0.05 level (2-tailed)

93

Table 11

Descriptive Statistics for Published Articles Subpopulations (n=360)

Variable Level

Freq. Percent aDSI (range) SD aDSW SD

Year of Publication 360 100.0 .11 .10 900 956

2011 111 30.8 .11 .11 900 944

2012 78 21.7 .13 .11 1139 1247

2013 76 21.1 .10 .10 804 772

2014 81 22.5 .11 .11 814 849

2015 14 3.81 .07 .04 593 294

Research Method 360 100.0 .12 .10 1022 958

Quantitative 122 33.9 .13 .12 1233 1220

Qualitative 83 23.1 .08 .09 727 741

Other 155 43.0 .10 .09 732 732

Author Count 360 100.0 .11 .10 900 956

1 122 33.9 .12 .10 903 917

2 95 26.4 .12 .11 1028 956

3 76 21.1 .13 .10 1177 1147

4 33 9.2 .14 .11 1138 871

5 through 18 34 9.4 .12 .07 975 690

Document Similarity Level 360 100.0 .12 .10 1022 958

Low 285 79.2 .07 (.00-.14) .04 576 423

High 46 12.8 .19 (.15-.24) .03 1588 721

Excessive 29 8.1 .38 (.25-.84) .13 2995 164

Published Article Differential Statistics Results


Turnitin’s reported document similarity indices (DSI) and the adjusted document


94

Researchers often use the paired t-test statistic to test for differences between

pairs of data. However, a paired t-test assumes data normality with the difference

between the two variables. Figure 15 employs two histograms, which displayed the

visual difference between the published article DSIs and the aDSI. The left DSI

histogram exhibited a slight bimodal distribution while the right aDSI histogram was

unimodal with extreme skewness. The histograms provided a visual indication of non-

normality distributions.

However, the t-test assumption of normality is based upon the difference

between the two dependent variables. The statistical difference between the DSI and

aDSI tested produces a skewness statistic of -.640 (SE = .129), Z = 4.96 and a kurtosis

statistic of -1.355 (SE = .256), Z = 5.29 (Cramer, 1998; Cramer & Howitt, 2004). The

histogram (see Figure 16) illustrates a bimodal distribution with this skewness. A

Shapiro-Wilk Test for Normality statistic was conducted on the difference (Shapiro &

Figure 15. Published article DSI & aDSI frequencies before and after adjustments. The black bars represent the Document Similarity Levels separators for Low, High, and Excessive.

95

Wilk, 1965; Field, 2011). The results returned a statistic of .792 (df = 360, p < .05). The

data tested violated the assumption of normality so a paired t-test would not be suitable.

Because of the confirmed abnormality of the data, I conducted a Wilcoxon

Signed-rank statistic (Zimmerman, 1996; Field, 2011). As previously discussed in the

Methodology section, the Wilcoxon Signed-rank statistics ranked the data sets and then

compared the differences between the two sets. The Wilcoxon Signed-rank test

confirmed the significance of the difference between the DSI and aDSI reporting Z =

-15.626 (p < .05). The Wilcoxon Signed-rank effect size estimate r value of .58 (p < .05)

Figure 16. Histogram exhibiting difference between dissertation paired DSI & aDSI.

96

provide strong support that these adjustments were statistically and practically

significant (Field, 2011; Rosenthal, 1991). Rosnow and Rosenthal stated a Wilcoxon

Signed-rank effect size estimate r value of 0.50 and above is large (2005).

Published Article Predictive Statistics Results




I also conducted a multinomial logistic regression (MLR) analysis to predict

published article membership outcomes in the DSL. The goal of an MLR analysis it to

determine what independent variables influence the probability of a document belonging

to one of the three DSLs (see Figure 17).

Figure 17. Published article membership in DSL groups.

97

Overall, within the corpus of 360 published articles, SRT identified 285 cases or

79.2 % in the low plagiarism range, 46 cases or 12.8% in the high plagiarism range, and

29 cases or 8.1% in the excessive plagiarism range. However, the document research

method “mixed” had only 4 cases in the high DSL and only one in the excessive DSL.

The research method “other” classification had 15 members in the high DSL and 12

members in the excessive DSL. Lee, Ahn, Moon, Kodell and Chen (2013) stated,

"Although logistic regression is known to be robust as a classification method and is

widely used, it requires that there be more observations than predictors" (p. 682).

With 360 observations, I have met that criterion. However, with such a high

concentration (separation Issues) in the low DSL, I again suspected, the prediction

would be difficult (Anderson, 1984). As previously noted, to improve the model I moved

the “mixed” research method cases into the "other" research method cases.

The Nagelkerke's Pseudo R2 is a model based pseudo effect size statistic used in

MLR to explain the importance of the predictors in terms that closely behave like a

linear model (Allison, 2013). Bewick, Cheek, and Ball (2005) explained using

Nagelkerke’s Pseudo R2 to demonstrate “how useful the explanatory variables are in

predicting the response variable and can be referred to as measures of effect size” (p.

116). With the available predictors arranged in all of the possible combinations, the

highest Nagelkerke's Pseudo R2 of .095 was realized using Research Method, Year of

Pub, Author Count, Word Count, and Reference Count as predictors (see Table 12).

98

Table 12

Modeling MLR Analysis of Sampled Published Articles

Model Classification Percentages Effect size (Pseudo R2)

Low High Exces-sive

Total Cox & Snell Nagelkerke Mcfadden

Research Method Year of Pub Author Count Word Count Reference Count

100 0 0 79.2 .069 .095 .055

Research Method Author Count Word Count Reference Count

100 0 0 79.2 .061 .084 .049

Research Method Year of Pub Word Count Reference Count

100 0 0 79.2 .061 .084 .048

Research Method Word Count Reference Count

100 0 0 79.2 .053 .072 .041

Year of Pub Author Count Word Count Reference Count

100 0 0 79.2 .049 .067 .038

Year of Pub Word Count Reference Count

100 0 0 79.2 .044 .061 .035

Author Count Word Count Reference Count

100 0 0 79.2 .041 .056 .032

Research Method Year of Pub Author Count Reference Count

100 0 0 79.2 .038 .052 .030

Word Count Reference Count

100 0 0 79.2 .036 .050 .028

Research Method Year of Pub Author Count

100 0 0 79.2 .037 .051 .029

99

Word Count

Research Method Year of Pub Author Count

100 0 0 79.2 .033 .046 .026

Research Method Year of Pub Word Count

100 0 0 79.2 .032 .044 .035

Research Method Year of Pub Reference Count

100 0 0 79.2 .031 .043 .024

Research Method Author Count Reference Count

100 0 0 79.2 .030 .041 .023

Research Method Author Count Word Count

100 0 0 79.2 .029 .040 .023

Research Method Year of Pub

100 0 0 79.2 .028 .038 .022

Research Method Author Count

100 0 0 79.2 .025 .035 .020

Research Method Word Count

100 0 0 79.2 .024 .032 .018

Research Method Reference Count

100 0 0 79.2 .023 .032 .018

Research Method 100 0 0 79.2 .019 .027 .015

Year of Pub Author Count Reference Count

100 0 0 79.2 .017 .023 .013

Year of Pub Author Count Word Count

100 0 0 79.2 .015 .021 .012

Year of Pub Reference Count

100 0 0 79.2 .013 .018 .010

Year of Pub Word Count

100 0 0 79.2 .012 .017 .010

Year of Pub Author Count

100 0 0 79.2 .012 .016 .009

Year of Pub 100 0 0 79.2 .009 .012 .007

Author Count Reference Count

100 0 0 79.2 .009 .012 .007

100

Author Count Word Count

100 0 0 79.2 .007 .009 .005

Reference Count 100 0 0 79.2 .005 .006 .004

Author Count 100 0 0 79.2 .003 .005 .003

Word Count 100 0 0 79.2 .004 .005 .003

Table 13 exhibits statistically significant odds ratios for word count and reference

counts in the excessive DSL. A continuous variable odds ratio reflects the change in its

odds ratio per one unit increase change in the continuous variable. In this study one

unit of change for words is 100 words and one unit of change for references is 20

references. For example, using these values for the references variable, one unit of

reference increase change (20 references) has an odds ratio increase of 3.666 that it

will belong to the excessive DSL over the low DSL as long as all other variables remain

constant. Regarding the word count variable, one unit of word increase change (100

words) has an odds ratio of .124 to 1, which interprets as that this change in words is

88% (1 - .124) less likely to remain in the excessive DSL over the low DSL as long as all

other variables remain constant.

In summary, Table 13 exhibits statistically significant odds ratios for word counts

and reference counts at only the excessive level. The model's overall practical effect

size using the Nagelkerke Pseudo R2 was .095. The practical effect size is very small.

101

Table 13

MLR Analysis of Sampled Published Articles (Best Model) n=360

Dependent Variable Levels Independent Variable

B Std Error Wald df Sig Exp(B)a

Odds Ratio

95% CI

LB UB

High Document Similarity Level Publication Year Author Count Word Count (x100)

Reference Count (x20) Qualitative Method Quantitative Method Other Methodb

482.227 -.240 -.061 -.592 .498 .273

-.015 .0

272.945 .136 .100 .434 .268 .381 .428

3.121 3.143 .369

1.857 3.443 .516 .001

1 1 1 1 1 1 1 0

.077

.076

.543

.173

.064

.473

.972

------ .786 .941 .553 1.646 1.314 .985

------ .603 .773 .236 .972 .623 .425

------ 1.026 1.145 1.296 2.785 2.770 2.281

Excessive Document Similarity Level Publication Year Author Count Word Count (x100) Reference Count (x20) Qualitative Method Quantitative Method Other Methodb

123.470 -.062 -.155

-2.084 1.299 .680

-1.019 .0

339.086 .168 .152 .557 .316 .450 .794

.133

.136 1.040

13.996 16.878 2.147 1.648

1 1 1 111 1 0

.716

.712

.308

.000

.000

.143

.199

------ .940 .856 .124

3.666 1.934 .361

------ .675 .636 .042

1.973 .800 .076

------ 1.307 1.154 .371

6.814 4.673 1.711

aConfidence Intervals for Exp at 95% bThis parameter is set to zero because it is the Research Method reference category.

Note: Comparison Category is Low Document Similarity Level. Bold signifies statistically significant.

Figure 18 provides visual representations in the form of scatter plots for both the

continuous independent variables word count and reference counts. What is noticeable is

the heteroscedasticity in both the word and reference count scatter plots. Moreover, just

as with the dissertation scatter plots for these two variables, it is evident that the

regression lines between the word and reference count independent variables are going in

opposite directions.

102

Figure 18. Published article scatter plots for word count and reference count variables. The green and orange lines divide the three DSL levels and the red line is the regression line.

103

DISCUSSION

This discussion section combined the two corpora and compared the analysis

results and the methodologies of this study with prior studies. These comparisons and

expanded explanations provided an overview of the memberships in each DSL for

dissertations and published articles. Moreover, I discussed the implications this study

has for stakeholders as well as the direction of future research. Additionally, I reviewed

various encountered issues and obstacles with the intent to help future researchers

avoid some of the mistakes I made.

Discuss and Synthesize Research Findings

The academic and publishing profession, with good reason, has yet to develop

an agreed upon criteria for evaluating plagiarism and content similarity values. First,

similarity values, by themselves, are not proof of plagiarism. Second, there is a political

overtone tied to adjudicating plagiarism, as evidenced by Moore (2008), a managing

editor for the Ventura County Star, when he stated, “We have zero tolerance for

plagiarism” (n.p.). Thus, publically there cannot be any acceptable level of plagiarism.

However, with limited reviewing resources (Willis, 2016), the application of plagiarism

levels can effectively allocate reviewer resources in the areas where they will be the

most effective in detecting plagiarism. Finally, there are no commonly accepted best

practices for detecting false positives. An examination of the Turnitin documentation and

prior studies did not provide any finite instructions that would guide reviewers or lead

researchers towards research “replicability” (Fournier, 2016, n.p.). This study provided

me with the opportunity to document both a set of specific processing steps for an

effective COA analysis and a set of synthesized similarity levels. Similarity levels can

104

serve institutions and publishers in identifying potential plagiarism events and the

appropriate remedies for their adjudication.

Descriptive Findings

For the majority of corpus plagiarism studies, descriptive statistics have been

instrumental in bringing a broad understanding to the phenomena of plagiarism (Honig

& Bedi 2012; Sun, 2012; Youmans, 2011; Batane, 2010; Keck, 2006). This exploratory

study examined two different corpora of documents that provided two sets of descriptive

data and the opportunity for cross checking. The Turnitin COA reports and derived data

provided a rich set of descriptive statistics that provided the DSI, aDSI, DSW, and

aDSW variables across the corpus of dissertations and published articles. This study

called attention to the synthesized word count variables DSW and aDSW because they

added a perspective that went beyond what the percentages alone provided.

The most important descriptive statistics gathered from this study were the DSI

and the aDSI. Corpus plagiarism studies often provide plagiarism data regarding

document plagiarism (similarity) by percentages or plagiarized word counts. However,

prior research has not been clear about whether the reported plagiarism statistics were

based on what Turnitin initially provided or if and how the investigator had adjusted the

Turnitin DSI down to an aDSI. I reported the DSI, the aDSI, the DSW, and the aDSW.

Using the DSI and synthesized DSW as my starting point, I found the aDSI statistics for

the dissertations (M = .09, SD = .06) was about 80% of the aDSI of the published

articles (M = .11, SD = .10). However, the aDSW for the dissertations (M = 3828, SD =

2484) was almost four times the aDSW to the published articles (M = 900, SD = 956).

105

These findings illustrate the importance of recording the number of content words in a

plagiarism study document.

In support, Ison (2012) suggested that the number of potential words plagiarized

is as important as the percent of the document plagiarized. Ison reported word

similarities based upon document words and the percentage of the document

similarities. Honig and Bedi (2012) also used plagiarized words as their primary statistic

over the percentage plagiarized for identifying important findings in their study.

While these findings infer that plagiarized word counts across various corpora

are important in COA research, researchers should examine the wide ranges in

document word counts within a corpus. For instance, my study recorded the smallest

article at 699 words (one page editorial with one reference) and the largest article at

24,463 words. The ratio difference (1 to 34) between those two articles is larger than the

mean word ratio differences (1 to 6) between the corpus means of the publish articles

(M = 8552, SD = 4186) and the dissertations (M = 48603, SD = 22833).

My study also collected 1193 instances of non-excluded dissertation SSI equal or

larger than 1% of the dissertation corpus and 2254 instances of the same from the

published article corpus. From that collection, I identified 14 dissertation plagiarism-of-

others substantive SSIs (M = .06, SD = .02) and from 167 published article instances, I

identified 42 plagiarism-of-other substantive SSI (M = .08, SD = .05) and 125 instances

of substantive plagiarism-of-self SSI (M = .12, SD = .09).

These figures lead to an important finding that my study identified. By far, after all

the false positives were removed, the majority of the individual SSIs were 4% or less.

Very few dissertations or published articles had substantive SSIs. Thus, the absence of

106

substantive SSIs leads one to conclude that most plagiarism issues are an

accumulation of small infractions.

Secondary findings from this study were derived from the document metadata in

each corpus. Publishing year, research methods, author counts, word counts, and

reference counts provided good statistics that institutions and publishers could use to

research and establish submission expectations. A breakdown of the documented

publishing years indicated that publishing years were well represented in HRD related

dissertations and published articles (see Table 6 and Table 11). However, one should

understand that the publishing year does not always represent the year of the study or

the date of authorship.

It was no surprise that these corpora predominantly employed quantitative and

qualitative research methods. However, while low, dissertations as compared with

published articles employed almost double the number of mixed-method research

designs. Moreover, published articles had a significant number of other (non-empirical)

articles (see Appendix A and B for the breakdown between each DSL). One should also

note that the quantitative research method designs had higher mean aDSI rates in the

corpora (see Tables 6 and 11).

Dissertations always had a single author, whereas this sample of published

articles listed author counts ranging between 1 and 18. With these results, this study

could only utilize author count as an independent variable for the publish article corpus.

While I have previously noted that dissertations had almost six times the number of

words that published articles had in the study’s data, one should note that dissertations

occasionally contained sizeable appendices, which added to the word count.

107

Published articles had more references per 1,000 words than dissertations.

Comparing the dissertation’s mean of 48,603 words with the mean of 140 references to

the published articles means of 8552 words with the means of 46 references, we have

dissertation with 2.88 references per 1000 words (r = .53) and published articles with

5.38 references per 1000 words (r = .77).

As previously discussed, this study defined a synthesized set of DSLs using

various prior classification levels as a foundation. Derived from Thomas and de Bruin

(2014), the Higher Education Commission, Pakistan (n.d.), and others (see Table 2),

these levels in Tables 6 and 11 identify the document counts, mean aDSI, and mean

aDSW in each of the three DSLs for dissertations and published articles.

This study examined a corpus of 360 dissertations using Turnitin and returned

88.1% of the dissertations in the low levels of plagiarism, 9.7% in the high level and

2.2% in the excessive level. Table 14 compares the results of this study to several other

important dissertation plagiarism studies.

Table 14

Dissertation Corpus Plagiarism Studies using Turnitin

Authors Year Corpus n aDSI (SD)

Low 0%-14%

High 15%-24%

Excessive 25%-100%

Mayes (Current) 2017

Online & Traditional HRD Dissertations

360 .09(.06) 88.1% 9.7% 2.2%

Isona 2014 TraditionalDissertations 184 .15(.13) 54.0% 28.0% 18.0%

Isona 2014 Online Dissertations 184 .14 (.08) 54.0% 36.0% 10.0%

Isona 2012 Online Dissertations 100 .15 (.13) 52.0% 34.0% 14.0%

aI estimated DSL Membership using interpolation techniques.

108

Ison (2012) studied 100 dissertations retrieved from ProQuest with publication

years from 2009 to 2011. Using Turnitin to analyze the dissertations for evidence of

plagiarism, he applied Ison’s data to Bretag and Mahmud (2009) similarity levels. His

analysis identified 40% of the dissertations with no or low plagiarism (DSI of 0 to .10)

and 46% of the dissertations with medium plagiarism (.11-.24). He also found that 11%

of the dissertations had high plagiarism (DSI of .25-.49) and 3% (DSI of .50 and above)

had excessive plagiarism.

Furthering his dissertation plagiarism research, Ison (2014) examined a corpus of

368 dissertations published between 2009 and 2013 for evidence of plagiarism. He

compared 184 dissertations from predominantly on-line programs to 184 dissertations

from traditional institutions again using Bretag and Mahmud (2009) similarity levels. In

both Ison studies, there were limited details about Turnitin configurations nor any details

about false positive exclusions.

The results show a sizeable difference between Ison’s and my study’s results.

Ison's results are consistent across all three corpora and are a good indication that

consistent methodologies produce consistent results. Ison described the procedures he

used in both of his studies to avoid or remove false positives:

Quotations and definitions were manually omitted from the analysis. Results

were examined for potential similarity overlaps with work previously submitted by

the author at the institution that awarded the doctorate. Such overlaps were then

excluded from the final similarity score used in this study (2012, p. 231-232).

Quotations, bibliographies, and definitions were omitted from the analysis. The

initial similarity indices were examined for potential overlaps with work previously

109

submitted by the author at the institution that awarded the doctorate and were

subsequently removed if applicable. (2014, p.276)

The important finding from my study’s dissertation descriptive statistics, when

compared to Ison's dissertation descriptive statistics, was that identifying the plagiaristic

activity with a corpus of dissertations is merely a snapshot and highly dependent upon

the various configurations and methodologies employed to reduce the false positives.

My study also examined a corpus of 360 published articles using Turnitin and returned

79.2% of the published articles in the low levels of plagiarism, 12.8% in the high level

and 8.1% in the excessive level. Table 15 compares the results of this study to several

other important studies on published article plagiarism.

Table 15

Published Articles Corpus Plagiarism Studies using Turnitin

Authors Year Corpus N aDSI (SD)

Low 0%-14%

High 15%-24%

Excessive 25%-100%

Mayes (Current) 2017 Published HRD

Articles 360 .11 (.10) 79.2% 12.8% 8.1%

Thomas & de Bruina 2014

South African Management journal articles

371 .17 (.12) 51.5% 27.2% 21.3%

Suna 2013 Published STEMArticles 300 .13 (.12)

64% 31% 5% Breakdowns between STEM and Social

Science articles were not published. Suna 2013 Published SocialSciences Articles 300 .08 (.08)

Honig & Bedi 2012 Presentation Papers 279 No comparable descriptive statistical percentages.

The study used word counts only.

aI estimated DSL Membership using interpolation techniques.

110

Thomas and de Bruin (2014) submitted a corpus of peer-reviewed articles

(N=371) to Turnitin for COA analysis. They reported corpus descriptive statistics that

found 51.5% of the published articles were members in the low group. The high

plagiarism group had 27.2% membership, and the excessive plagiarism group had

21.3% membership. These researchers provided a detailed description of their

methodology for removing false positives:

The results for each article were checked twice and a conservative approach was

adopted in the interpretation of the similarity indices, in which the benefit of doubt

was in favour of the authors. For each article, the following content was not

included in the assessment of similarity: bibliography/list of references,

quotations, strings of words of less than 10, student write-ups on which the article

was based, conference proceedings and abstracts detailing the main features of

the article. In addition, during the second inspection of the data, specific

methodological terms and statistical or mathematical formulae were excluded in

the analysis of similarity. (Thomas & de Bruin, 2014, p.3)

Sun (2013) examined 600 articles, also using Turnitin, and reported membership

in six different DSLs. Sun excluded all similarities less than 30 words to avoid trivial

similarity issues. I experimented with Sun's 30-word exclusion algorithm and found it

greatly reduced the number of paraphrasing issues, which could have been counted as

plagiarism. Sun further described the method used to remove additional false positives:

A manual check was employed by the researcher of this study to make

qualitative judgments on the appropriateness of textual re-use. In the current

study, human screening of each match was conducted across 4247 matches and

111

the following types of matches were excluded from the results: (1) the exact

article found in the Turnitin database, (2) text in quotation marks or displayed

quotations, (3) formulae and terminology, and (4) article titles and author

information. (2013, p. 267)

Honig and Bedi (2012) examined 279 papers by 636 authors who presented at

the International Management Division of the 2009 Academy of Management

conference. In describing the methodology for removing false positives, they stated:

“We manually checked the highlighted sections for appropriate citations and excluded

the methodology section of all empirical papers” (pp.112-113). They reported that

25.44% of the corpus (71 papers) they reviewed had some level of plagiarism and that

13.6% of the corpus (38 papers) had significant levels (5% or more of plagiarized

content).

My attempt to compare my study with these studies demonstrated that there is a

lack of conformity and continuity between all COA corpus studies. However, the current

accumulation of plagiarism research affirms that both the incidence of plagiarism and

the method of researching plagiarism are serious issues, which need addressing.

Differential Findings

Software like Turnitin is best suited for detecting potential plagiarism on

unpublished manuscripts. That being said, using plagiarism detections software on

previously published corpora, while requiring difficult and time-consuming processes,

can be effective. However, finding and excluding false positives are crucial to the

accuracy of a COA. While this process is person-power intensive, if COA results are to

be meaningful (valid and replicable) the process must be well defined and executed with

112

sufficient adherence to the process. However, there is little in the literature that provides

definitive instruction. Turnitin indicated that COA reports are just a starting point, and

that investigator intervention is required. They provided no statistics that expressed the

importance of the intervention or how the investigator should execute the intervention.

An important finding from this study is that there is strong statistical support for

the removal of false positives in Turnitin COA reports. My research determined that the

differences between the initial DSI generated by Turnitin and the aDSI I calculated are

statistically and practically significant. The Wilcoxon Signed-rank statistics revealed a

statistically robust and practically significant difference in the corpus of dissertations

before (DSI) and after (aDSI) variables, p < .05 with a large Wilcoxon Signed-rank effect

size (r =.52) and in the corpus of published article before (DSI) and after (aDSI)

variables, p < .05 with a large Wilcoxon Signed-rank effect size (r =.58). The findings

from my study’s differential statistic tests demonstrated that for both corpora there was

a statistically and practically significant need to investigate and remove false positive

SSIs from the COAs. Bypassing this process on either corpus would render COA results

invalid and meaningless. These results confirmed the need for the removal of false

positives.

In support, Batane (2010) reported that he found when using Turnitin, the COA

report suffered from a “tendency of the software to identify material as plagiarized” (p.

3). He suggested that Turnitin users verify all instances of identified content similarities

and make the necessary adjustments. Braumoeller and Gaines (2001) in a study using

a software called EVE, noted that false positives were an issue that clouded their

plagiarism detection analysis. They found that almost 50% of the papers flagged, had

113

been properly cited and referenced. Sun (2013) in her study seemed to reduce the

issue of false positives by using an exclusion for similarities of less than an arbitrary 30-

word cut-off point. Sun concluded that " the quantitative findings of the current study

indicate that authors in contexts wherein English is an official language do not differ

significantly from their counterparts on their Turnitin scores or the number of 30-word or

longer strings of successive matching text from self-published articles and self- and-

others’ publications combined." (p.270). Recounting my experience testing a 30-word

exclusion, I found that this configuration eliminated many sentence size instances of

paraphrased plagiarism. Jocoy and DiBiase (2006) performed manual document checks

to identify false positives and used an aDSI for their analysis. Martin, Rao, and Sloan

(2011) individually removed all cited, quoted, and bibliographic references and manually

checked for potential false positives.

Honig and Bedi (2012) reported that to reduce the number of false positives that

they manually checked the highlighted areas within the manuscripts for false positives.

Moreover, they excluded the methodology and reference sections of all empirical

papers. Regarding plagiarism-of-self, they stated:

Since the focus of this study was on individuals plagiarizing others’ work without

appropriate acknowledgment, we adopted a more conservative approach toward

self-plagiarism. If authors used sections from their own previous work or cited the

primary source, then it was not considered plagiarism. (p.112)

114

Prediction Findings

My study focused upon five predictors derived from the available metadata

gathered from the two corpora. Being that participation in plagiarism activity is a

personal decision, I did not expect any strong prediction based upon document

metadata. However, this study did find that the dissertation or publish article metadata

provided some evidence of plagiarism prediction. Using multinomial logistic regression

statistics, I determined that the optimal model Nagelkerke Pseudo R2 for the

dissertations was .169 and for the published articles was .095. Moreover, the

independent variables that had statistically significant values (odds ratios) were word

counts and reference counts. First, increases in reference counts predicted increases of

plagiarism, while increases of word counts predicted reductions in plagiarism.

Most of the other plagiarism prediction research was based upon controlled

interventions, human demographics, topic knowledge, and situations. Braumoeller and

Gaines (2001) used a software called EVE to conduct a plagiarism study using student

assignments with and without student warnings explaining plagiarism detection. While

they concluded that “The results of plagiarism tests should not be taken to be definitive”

(p. 836), they also stated that “At this stage, plagiarism-detection software is useful” (p.

836). While obtaining inconclusive plagiarism statistics, they predicted that just warning

students about plagiarism detection seemed to make students more diligent in avoiding

plagiarism activities.

Jocoy and DiBiase (2006) also attempted to predict the outcomes from a student

intervention but used Turnitin for plagiarism detection instead of EVE. Having measured

plagiarism rates on their students’ first assignment without the students having any

115

knowledge of Turnitin, they then provided a Turnitin demonstration and warned about

the consequences of plagiarism. When Jocoy and DiBiase evaluated the second

assignment, they suggested that they found a statistically insignificant, but measurable

drop in plagiarism rates.

Ling (2006) researched the premise that L1 (English as the first language) and

L2 (English as a second language) students had different perceptions regarding

plagiarism. Having conducted semi-structured interviews with 46 participants, she

concluded that the results were mixed. She also found that citation experiences varied

from participant to participant and that subjects from various cultures believed that

words are to be shared by all and not owned. Moreover, her research found that L2

students viewed the science of citations, paraphrasing, and plagiarism as hurdles to

their authoring skills.

Hege (2008) found that the affective state of mind could predict the ability to

recall an idea’s source. She applied her research to plagiarism and implied that those

authors in a good mood are more likely to forget the source of the ideas they are using.

Thus, accurate citing might be impaired.

One of the most interesting studies on prediction is Batane’s (2010) student

intervention study; it is similar to the Jocoy and DiBiase (2006) study. Measuring COA

rates with Turnitin on the first assignment from 272 students who were unaware that

Turnitin was being used to check their work, the students’ mean aDSI was reported at

.20. Assignments with evidence of plagiarism were downgraded and returned to the

students. After introducing the students to Turnitin and stating they were using the

software to check their second assignment, the reported mean aDSI was reduced by

116

.04 to .16. Moreover, in a category measure used by Batane, labeled as legitimate

research, measured at an aDSI of zero, the percentage of documents at that level rose

from 14.4% to 52.1%.

Martin, Rao, and Sloan (2011) studied 158 students using one specific

assignment submitted by each student to Turnitin. The focus of their study was to

predict plagiarism based upon participant demographic data. They used ethnic markers

for Caucasian and Asian students. They found no differences in predicting plagiarism

that ethnicity could explain. However, heritage or acculturation or the time spent in a

culture did show some linkage to plagiarism. Using a Manova Multivariate Test (Wilks'

Statistic) they identified an overall moderate effect size of ƞ2=.11 (p=.037).

Honig and Bedi (2012) had access to several author related demographics in the

course of a submission process at the 2009 Academy of Management conference. They

used this information in an examination of plagiarism prediction across 279 documents.

They wanted to know if plagiarism was affected by an author being a tenured or senior

scholar, by L1 or L2 authors, by an author’s country, by an author’s country being

established or matured, and finally by an author’s gender. They found no statistical

support for prediction concerning an author being a tenured or senior scholar or the

author’s gender. However, they did find statistical support for predictions for gender as a

moderating force with an author’s country being newly established or less matured

(more plagiarism), and between authors with L1 or L2 (more plagiarism) backgrounds.

Sun (2013) configured Turnitin for a 30-word exclusion for checking content

similarities and found that the number of authors did influence plagiarism rates (the

single author was the lowest). With her configuration at ten times the default Turnitin

117

setting, she found that an author’s official language did not influence plagiarism rates.

Ison (2012, 2014) conducted dissertation corpus studies and explored the premise that

on-line dissertations would have higher plagiarism rates than more conventional

institutions. He did not find any evidence that supported these hypotheses.

Given the various, sometimes conflicting evidence of prediction, it remains clear

that better, more defined techniques need to be developed. Plagiarism corpus studies

are very complex. The slightest variance in strategies and techniques can have a huge

impact on the reported results. However, the constant that seems to reappear is that

educating the student or author about plagiarism and its detection does influence

outcomes.

Discuss Document Similarity Levels

Excessive Levels of Plagiarism

Across the corpora of 360 dissertations and 360 published articles, this study

experienced 37 documents in the excessive DSL (25% - 100%). Rounding out the ten

highest plagiarized documents in this level of plagiarism were published articles where

large blocks of content came from prior articles or book chapters from the same

author(s). Often the abstracts were identical, and the titles were similar. Sometimes the

author lists varied. For instance, three authors instead of two, or authors listed in

different orders were common variances. The articles were normally published by

different publishers. However, the articles appeared to be submitted around the same

time to different journals, but more often than not, the final acceptance dates were

different.

118

Another common type of plagiarism-of-self at this level was covering the same

topic using different approaches. Occasionally, I found an article proposing a theory and

design for a particular type of study, then another article for the presentation of the

completed study, then a follow-up article using the same data but employing group

breakouts. Also, there were authors who published additional articles that discussed the

validity of the prior studies. Most of the articles at this level also had some serious

paraphrasing issues (plagiarism-of-others), but by far plagiarism-of-self issues

dominated the articles at the excessive level. While there are valid reasons for creating

a series of articles, it is critical that the readership is aware of the potential duplications

and that publishers have granted permissions to revisit prior published research or

articles.

Dissertations belonging to the excessive DSL had serious paraphrasing issues in

plagiarism-of-others and no plagiarism-of-self complications. Turnitin identified

paragraph after paragraph, borrowed from other authors, where the researcher had

changed only a few words. The number of SSIs was often as high as 200. However, the

amount each SSI contributed was normally less than the 5% threshold.

I did find one instance where a Ph.D. candidate had authored a published article

and plagiarized it in their dissertation that followed. This SSI was calculated at 4%. As

more and more doctoral candidates publish, this phenomenon may become more

common. At the excessive DSL, published articles and dissertations would need major

rewriting, probably be rejected, and the candidate failed. Furthermore, the ethical

implications should be investigated by the publisher, or in the case of a dissertation, the

institution.

119

High Levels of Plagiarism

Across this corpora of 360 dissertations and 360 published articles, this study

found 81 documents in the high DSL (15% - 24%). Published articles and dissertations

equally dominated membership in the high DSL. Paraphrasing issues were the

dominant problem. Having sentence size blocks of text without double quotes remained

a serious issue with all documents at this DSL. More often than not, the content

similarities were cited, but there are no double quote marks or indented paragraphs that

indicated these were not the author’s words. Several of the articles used single quote

marks at this level causing Turnitin to identify the content in the SSI and DSI. However,

there were still many instances of paraphrasing that influenced the DSI.

At the high DSL, it is impractical for published articles and dissertations to be

corrected without major rewrites. However, the author would be faced with rewriting the

manuscript, paragraph-by-paragraph, adding, or retaining required citations, and

possibly quotation marks. An author who does not want to improve their attempt at

paraphrasing would have to consider whether putting so much of the document in

double quotes would leave a reader wondering what was original. However, this is a

chance for institutions and publishers to influence an author’s research and writing

habits. Providing guidance at this point could correct future issues and have an effect on

the quality of research being published. As I was examining one high-DSL article, it was

discovered that the author had not plagiarized. He had written a paper he published in

the ERIC database and had borrowed part of it for a recent article. According to ERIC's

(2017) submission policies:

120

When you contribute your work to ERIC, you grant permission to index the

material and disseminate it online. You do not transfer copyright to ERIC and

may seek publication. (n.p)

When I excluded that source, it descended into the low DSL. This illustrates the due

diligence an investigator must possess to evaluate each instance of similarity.

Low Levels of Plagiarism

This study examined 602 documents out of the 720 documents across the two

corpora that belonged to the lowest level of the DSL. The reader may find interest in

that many of the published articles and dissertations in the lowest levels of plagiarism

(0% - 14%) had some of the highest starting Turnitin DSIs. I commonly found that the

first one or two exclusions (false positives) removed almost all of the reported content

similarities. Several cases of online institution dissertations had several committee

members repeatedly submitting these dissertations to Turnitin at various institutions.

Often Turnitin noticed this and reported DSIs well over .9. One can see the seriousness

of these issues as they show up in the histograms as this phenomenon created the

starting bimodal distributions (see Figures 9 and 12).

I found that it was common for short essays and editorials in our random sample

of published articles to have low levels of content similarities. While the ideas may not

be solely the author’s, the content was written in their words. Often there were a limited

number of citations and references. Overall, having 602 out of 720 documents in the low

level is a very positive outcome for the field of HRD.

121

Issues and Obstacles

I want to re-emphasize that Turnitin was not engineered for corpus wide COA

studies using published documents. However, this study will include discussions points

concerning potential feature changes or additions. That being said, many of the corpus

COA issues in this study seemed to be from inadequacies in the understanding of the

Turnitin software design and use. A Google search for a “Turnitin technical manual”

produced 415,000 results starting with “Getting Started, Student Manual, Instructor

Manual” and “User Manual.” There was little, which technically explained what happens

and how and why Turnitin does certain things.

For instance, in this study, I chose to bypass all similarity groups consisting of 10

words or fewer. A Turnitin anomaly may be that the 10-word exemption does not always

work. Below is an example where Turnitin flagged six words for inclusion in the DSI.

One can certainly see that this content is trivial in nature and does not meet the ten-

word exclusion (see Figure 19).

A common Turnitin setting removes quoted content from the DSI calculations.

However, as previously stated, this study found published papers where single quotes

were used for quotations, so Turnitin counted those direct quotations in the DSI.

Moreover, I found that DSI values of .01 - .02 were often a re-publishing of the title,

Figure 19. Trivial similarities of six words using a 10 word exemption.

122

abstract and the table of contents (TOC-dissertations only) across many different

marketing or reference sources. These marketing republishings created a monumental

task to exclude those SSI that referred to titles, abstracts, and tables of content.

Another challenge confronting corpus COA studies is how to address layered or

overlapping similarities. These are similarities, which exist across a multitude of

documents (Tucci & Galwankar, 2011). Several times, I had to delete a multitude of

SSIs, one at a time, just to reduce a DSI by .01 or .02. As I excluded these SSIs,

Turnitin produced more SSI for the submitted document with the same false positives.

These similarities were often trivial things like government agency names and

addresses, or institutional or publisher contact information, survey identification, and

even topic names. Even more troubling is that an SSI may have a combination of valid

and false positives. Additionally, an SSI can include plagiarism-of-others and

plagiarism-of-self. Turnitin has no apparent means to internally remove part of an SSI or

separate out and document those differences.

Often when the DSI was .90 or larger, the investigation pointed to duplication of

documents. However, I found several documents (primarily published articles) where

the only portion of the readable text was a standardized copyright notice use by a large

publisher. The main part of the document was a scanned image. Turnitin reported a

1.00 DSI because of the copyright notice, which had been published in hundreds, if not

thousands of other documents, was the only part of the document Turnitin could read. I

employed Adobe Acrobat’s image-to-text OCR feature to resolve this type of issue

before resubmitting the document to Turnitin.

123

During the dissertation corpus COA, I observed that most of the dissertations

were present in full or in part on the Internet at various locations. To name a few,

Internet URLs such as gradworks.umi.com, search.proquest.com, media.proquest.com,

www.coursehero.com, and www.researchgate.com provided copies of the same

documents or abstracts and links to the same document. I commonly experienced .01 to

.02 of the DSI were often linked to gradworks.umi.com, search.proquest.com,

media.proquest.com and were not instances of plagiarism, but of title or abstract

listings. The use of Turnitin at multiple institutions created large similarity issues.

However, often these issues stemmed from the same reviewer working at different

institutions repeatedly reviewing the same documents.

Furthermore, while reviewing the dissertations, it was possible to find similar, if

not identical dissertations under two different author names. Upon careful examination, I

realized the issue was not always a plagiarism issue because only the last name had

changed. The preponderance of the evidence suggested the change was a result of

marriage, divorce, or other legal name change event.

One of the dissertations had most of the references highlighted as Turnitin

content similarities. There was no internal way to exclude those Turnitin false positives.

I concluded that the attached appendices may have cause Turnitin to include the

reference list in its COA report despite its configurations instructing it to ignore reference

lists. Based upon a 100-page count for content and nine pages of references, I

estimated that by eliminating the anomaly, the aDSI would be reduced by .10, still

leaving the dissertation membership in the excessive level. Another dissertation was

authored with references included in each section (multiple article format). Again, the

124

format confused Turnitin, leaving it to identify the reference lists as content similarities

with extensive overlapping. To avoid these issues, it would be best to delete reference

sections before a Turnitin submission.

A minor issue that I need to mention is that sometimes an article shares pages

with the beginning or end of other articles. These submissions to Turnitin produced a

COA report on all of the submitted text. An investigator would be required to separate

out the similarities generated by the pieces of the adjoining articles.

While this study did not include the Turnitin student DCD in this study, I

performed a preliminary examination of its functionality. Reported SSIs tagged as

Student papers were most difficult to confirm as infractions. Turnitin does not make the

student papers available for inspection without permission from the person who had

submitted them to Turnitin. Moreover, when I attained the requested permission, I found

that students/authors often reused parts of their own previously submitted University

assignments in their dissertations. This practice is not plagiarism-of-self if they own the

copyright to their unpublished works. However, it may be against their university policy,

which is beyond the scope of this study. Furthermore, this practice is normal, as many

students are encouraged in their academic endeavors to select an area in their studies

and develop their expertise by authoring assigned essays and literature reviews.

Moreover, I found several examples where students had attended multiple schools and

used their prior research and writings repeatedly, thus creating many content similarity

issues. If the student or instructor had submitted these documents to Turnitin, it would

place the content in the separate student DCD, thus increasing the number of SSIs.

125

A serious logistics issue that affects corpus COA studies is the ability to track the

steps completed and record the plethora of resulting data. Spreadsheets are barely

adequate for tracking the processes and collected data. Researchers should use

database management skills in Corpus COA studies. Modern database management

systems have the ability to “drag and drop” develop user data input screens, and the

ability to query and export data from multiple tables in a database. This functionality

exceeds spreadsheet applications’ capabilities. However, database competencies are

skills a researcher may find difficult to learn.

That being said, I posit that software such as Turnitin should have the ability to

organize by project and save the corpus documents and reports, track all of the steps in

the process, save and display additional data that is critical to a COA, and provide an

accommodating query system.

Conclusions

While a review of the literature confirmed that plagiarism has been the focus of

many studies, ongoing research is still required. The literature does not indicate that

plagiarism levels have significantly dropped over the years. In fact, with the Internet,

plagiarism is incredibly easy and a time saver for both students and authors.

In response, this exploratory study documented a content originality analysis

(COA) of HRD-focused corpora of dissertations and published articles. Using Turnitin

COA software, this study identified and analyzed potential plagiaristic activity. Moreover,

this study documented the process of data collection and data analysis including

validating investigator-made adjustments to the initial Turnitin-reported results (see

Figures 2 and 8).

126

This study expanded the scope of COA descriptive statistics by using Turnitin-

reported variables, researcher-derived variables, and document metadata such as the

year of publication, research method, author counts, word counts, reference counts, and

subpopulation variables when applicable. However, I posit that the most important

descriptive statistic in a corpus plagiarism study is the DSL group membership based

upon the aDSI. A three-level DSL used in this study was engineered upon a foundation

of prior research studies. The aDSI values assigned membership in the low, high, or

excessive levels of the DSL. This study examined 360 dissertations and 360 published

articles and returned 88.1% of the dissertations in the low level of plagiarism, 9.7% in

the high and 2.2% in the excessive level, while 79.2% of the published articles were in

the low level of plagiarism, 12.8% in the high and 8.1% in the excessive level. The

differences in the rates between the two corpora can primarily be explained by the

incidences of plagiarism-of-self that were absent in the dissertations and present in the

published articles. Upon further examination, the aDSI means between the dissertations

and published articles of the low DSL (level 1) is .07 versus .08 (a positive 14%

difference). The high DSL (level 2) is .18 versus .19 (a positive 5% difference). The

excessive DSL (level 3) is .30 versus .36 (a positive 20% difference). The conclusions

that one can draw from the descriptive results from corpus plagiarism studies is that

these statistics alert us to ongoing plagiaristic activity within academic institutions,

publishers, and specific knowledge domains.

While I found little evidence that prior research had statistically identified the

importance of detecting and documenting false positives, the handling of false positives

proved critical to the results any investigator uncovered. For example, Dee and Jacob

127

(2012) analyzed 1279 student submissions. After a preliminary examination, they

reported 50% of the assignments had false positives. However, as with many studies I

reviewed, there were few details about the processes used for detecting these false

positives, nor any statistical and practical significance to what the identified false

positives played. In an attempt to define a previously undefined process such as in

Dee's and Jacob's (2012) work, I provided detailed process descriptions and evidence

of statistical and practical significance in defining the difference between the Turnitin

DSI and my aDSI. The dissertations started with a DSI of 27% and ended with an aDSI

of 9% or an 18% difference. The published articles started with a DSI of 67% and ended

with an aDSI of 11% or a 56% difference. One must conclude that an investigator

adjustment process is just too important to omit. Moreover, measuring its statistical and

practical significance adds an important metric to measure the accuracy of COA

studies.

Another important part of this study was identifying the difficulty for plagiarism

prediction. Using a multinomial logistic regression statistic, I found that there was a

statistically significant amount of prediction between the number of document

references and DSL membership. However, the interpretation of this finding is that

instructors and editors can assume documents with high reference counts would have

higher plagiarism issues. My research demonstrated that higher reference counts often

mask numerous cited paraphrasing issues lacking quotation markings.

The experience I gained from this study and the review of the literature leaves

me to believe that plagiarism prediction is still in its infancy. However, as our society

128

collects more and more data on individuals, there exists the potential to more accurately

predict plagiaristic based upon what may seem like unrelated data.

Implications

Academic and Publishing Domains

There is an overwhelming need for the academic and the publishing professions

to develop a set of best-practice strategies for use in future corpus plagiarism studies.

Zhang and Jia (2012) surveyed editors on what they perceived as troubling levels of

plagiarism and found that the

The majority of respondents indicated that if between one-quarter [25%] and one-

third [33%] of the content in the abstract, introduction or discussion is copied

without citation, the paper is likely to be rejected. (pp. 296-297)

Masic (2012) posited that if 25% or more of an article is not original, a publisher

should take remedying action. Samuelson (1994) reported that her colleagues used a

30% acceptance rule for plagiarism-of-self. Given these variations, it is clear that more

work needs to be completed. Most important, the profession should enact a set of

commonly accepted plagiarism levels. How institutions use these levels will be a matter

of individual preferences based upon publication goals and publisher resources.

However, academia and the publishing industry need to agree on some measurement

system. While the stated plagiarism goal is often defined as zero tolerance, the practical

application could utilize a three level plagiarism model related to the manuscript

submission outcomes of 1) accepted, 2) accepted with editing, and 3) rejected.

129

Professors and Instructors

While this study did not study plagiarism issues for undergraduate students, it

provided a basis for teaching all students about the science of plagiarism detection and

prevention. The key seems to be the need to keep the plagiarism discussion in the

forefront. Bailey (2016) writes:

Why don’t many students understand plagiarism? It’s simply because they were

never taught citation. Caught between instructors who thought it was “too early”

or “too late” to teach citation, they never really learned the art and never had its

importance impressed upon them. (p. 6)

Dee and Jacob (2012) also emphasized the importance of teaching students

about plagiarism prevention and from their research concluded that:

Our results demonstrate that a short educational tutorial can sharply reduce the

prevalence of plagiarism. The costs of this intervention are quite modest,

suggesting it could be scaled easily. It involves very little instructor involvement,

requires only 15 minutes on the part of students and the tutorial itself is freely

available. Moreover, our evidence suggests that the intervention has the largest

impact on lower-ability students, which may make it even more beneficial at a

wide range of public and private institutions with less selective admissions than

the highly selective institution we study. (p. 427)

Applying the research from Bailey (2016), and Dee and Jacob (2012) I instituted

the following learning aid for use in an undergraduate class, prefacing course essay

assignments:

How do you write without creating plagiarism issues? EASY!

130

• Read all your sources that apply to your essay. Take notes.

• Create your title and subheadings. Create the reference section in your paper

(You will continually need to edit the list by adding to it and deleting unused

ones.)

• Now write your article in your own words with your understanding of the

materials without directly looking at your sources. (DO NOT COPY, PASTE

and EDIT)

• Next, cite any ideas you found in your sources that are not yours or more

importantly, belong to others. You do not have to cite every sentence.

• If you want to add some exact quotes, cite and include a page number.

Example 1: Mayes (####) stated that "Small quotes are just placed within the

text using double quote marks" (p.45).

• Example 2: Mayes (####) also added that:

Large quotes, such as a whole paragraph, need to be separated out and

indented. Double quote marks are not needed. However, do not go

overboard on quotes. You want the reader to know you are the author and

not just copying the bulk of the text. (p. 46)

• If you use an image, you should list its origin and cite it.

• Note: You can cite a single reference several times.

A most effective way for students to become involved in plagiarism prevention is,

if possible have students submit their writings to Turnitin for a COA report. The process

leaves a student very aware of the issues and provides ample deterrent regarding

plagiarism. This study added to what other studies have posited, that under active

131

supervision students and authors avoid plagiaristic activity if they understand the

nuances of plagiarism. More important, their professors and instructors must lead by

attitude and example.

Researchers and Authors

While plagiarism-of-others is an institutional matter often driven by tenure-related

publishing counts, one has to recall what Rubio (2013) reported regarding a Columbian

Supreme Court case where a professor was sentenced “to two years in prison plus

monetary and civil sanctions for plagiarizing a student’s thesis” to fully understand the

seriousness of plagiarism-of-others (p. 141).

Regarding plagiarism-of-self, this study provided evidence of substantive

plagiarism-of-self SSI in published articles that explains the overall difference between

the aDSI statistics between published articles and dissertations. However, while there is

much discourse about the implications of plagiarism-of-self issues, given what

happened to Schminke and Ambrose (2014), an author, even a very renowned author,

cannot simply dismiss plagiarism-of-self matters. Plagiarism-of-self inflates the

importance of work and can exaggerate the contribution to a science. It is important that

a researcher and author notify readers of previous works and get copyright releases

when needed. Moreover, researchers and authors need to come to terms on what are

the ethical definitions for adjudicating plagiarism-of-self issues.

Reviewers and Publishers

Publishers control the ethics of plagiarism by the collective determination of what

is acceptable for publication. However, publishers need an industry-wide defined

process for detecting plagiarism. Plagiarism detection efforts cannot be discriminatory

132

and must be supported by policy and procedures that are verifiably consistent and

accurate. If they use peer reviewers to search for plagiarism, the publishers need to

provide the reviewers with tools, as well as the training, and the needed support when

plagiarism is detected. Most important, any adjudication of plagiaristic manuscripts must

include input from the author(s). It is most important for the author(s) to be given the

opportunity to defend themselves.

COA Software Providers

This study demonstrated that plagiarism detection software is only a tool that a

researcher can use to investigate potential evidence of plagiarism. Turnitin was

primarily engineered for examining unpublished documents that have not yet had the

chance to spread throughout the Internet. During the execution of this research, it

became apparent for published corpus research that Turnitin required some additional

features integrated into its design. Below is a list of suggested features and justifications

for the suggested additional feature requirements.

1. Show year of publication or Turnitin acquisition date with each listed similarity

source. It is critical to know whether the test document or the source document

was published first.

2. Display article title, authors, and publish date from internet sources as Turnitin

does with publications. This information is essential in a COA.

3. Be able to exclude headers and footers. Headers and footers are often publisher

generated across a vast number of documents thus leading to false positives that

are difficult to eliminate.

133

4. Be able to exclude selected text from the report. Eliminating selected text from

the report is critical for eliminating trivial and template false positives across all

sources.

5. Be able to exclude document title pages, copyright pages, abstracts, and

dedications. Common research documents and surveys, and institution and

publisher agreements generate false positives that are difficult to eliminate, i.e.,

set page ranges for content analysis at the COA report interface.

6. Be able to exclude document reference lists and appendices, i.e., set page

ranges for content analysis at the COA report interface.

7. Allow for a search of the DCD text stream for finding author name, Copyright

notice, etc. The DCD text stream is difficult to navigate, read, and interpret. The

ability to search specific words would help with a researcher's analysis.

8. Allow for copying of the text DCD stream. When material similarities occur, proof

must be documented to assist with a plagiarism resolution. Moreover, data

streams can change leaving once available evidence missing.

9. Allow for global exclusion configurations of common internet sources like

www.reseachgate.com, www.coursehero.com, www.DocStop.com,

www.Proquest.com, www.gradworks.umi.com, www.thefreelibarary.com and

www.slideshare.net. These websites often publish existing documents, in full or

in part, creating large and frequent false positives.

10. Provide counts on total exclusions with total percent excluded. It is important to

know how many false positives have been removed and what percent is

134

attributed to each excluded source. Moreover, it will be easier for a researcher to

restore exclusions for a re-investigation.

11. Provide an option to exclude all sources (sub-sources) under a certain

percentage, including those less than 1%. Hundreds of small similarity sources

often consisting of trivial and template similarities lead to false positives and are

difficult to eliminate. A blanket exclusion would free up resources for a more

thorough investigation on the larger similarities.

12. Option to exclude by ratio the number of source similarities that will not be

examined. Using an interpolation calculation, Turnitin can be set to estimate the

potential exclusions by the ratio of confirmed exclusions to the starting document

similarity index.

13. The exclusion “Restore All” button should provide the summative number and

percent of the excluded SSI that would be restored.

Future Research

While it is important to test the validity (replicability and accuracy) of this study,

researchers should also look to improve on what this study has already offered. I

encourage others to follow-up on this study's techniques and the analysis processes

employed, as well as the results. Because this study only covered dissertations and

published articles, additional work is required. Covering student plagiarism and cheating

is a continual challenge. Changing technologies and attitude shifts across students and

faculty require new research and a revisiting of prior research of plagiarism and

plagiarism prevention.

135

Future studies can be undertaken to determine if the processes and procedures I

documented to remove false positives are applicable with other plagiarism packages.

Furthermore, while I provided a short synopsis of several plagiarism detection

packages, it is important to see how the Turnitin results from this study would compare

using other packages. These studies could compare features, results, and usefulness of

competing plagiarism detection packages.

Additional COA studies can be used to compare results across different

professions and cultures. These studies can look for variations in the processes and the

results. Moreover, COA studies on unpublished documents could be compared to

published documents noting the statistical differences in needed adjustments and

results.

Another area needing review is the calculated final adjustment. This study use a

formula (see pages 64-65) to exclude a portion of the SSI less than 1% based upon a

ratio of previously excluded SSI. A random sample of the SSI less than 1% could be

used to verify the broader application of the formula across all SSI smaller than 1%.

This study defined substantive SSI as have similarity index of 5% or more.

Research should examine that percentage and verify the practicality of the 5%

threshold. Further research could examine the prospect that a word count threshold

would be more appropriate.

Additional validation research is needed regarding the usefulness of the set of

DSLs used in this study. My DSLs were based upon merging several existing levels

from other similar studies. However, my document similarity levels requires more

research to affirm that these levels are appropriate and could serve as a valuable set of

136

metrics in the academic research and publishing fields. A greater understanding of what

would be consider low, high, or excessive plagiarism is important. These studies could

use respected professors and editors to review sample documents that fit within the

three levels. They could identity what level best described the aDSI from these samples.

Either a quantitative or qualitative analysis of recorded impressions could potentially

confirm or adjust the DSLs with the most appropriate aDSI ranges.

Most important, the goal of plagiarism science must be the prevention of

plagiaristic activities. Further research can examine which educational techniques are

the most effective in reducing student plagiarism using various plagiarism detection

methodologies and preemptive intervention strategies. The publishing profession must

also identify and rank the best practices for prevention strategies based upon replicable

submission results.

In closing, our profession cannot overlook the importance of recognizing the vast

amount of original work that is the norm and avoid focusing on the vindictiveness of

punishing the ignorant. Plagiarism preventative measures will help our industry more

than punitive punishments. It is important to recognize that content originality is more

important than content similarity and that education is the key.

137

APPENDIX A

DISSERTATION FREQUENCY DETAIL TABLES

138

DSL Frequency Table for Dissertations with Research Method Subpopulation (n=360)

Document Low Similarity Levels 0%-14% 15%-24% 25%-100%

Freq. % Freq. % Freq. %

Quantitative 110 34.7% 22 62.8% 6 75.0%

Qualitative 149 47.0% 10 28.5% 1 12.5%

Other 58 18.3% 3 8.7% 1 2.5%

Descriptive Statistics for Dissertations Subpopulation (n=360)

Number of Wordsa Freq. Percent aDSI SD aDSW SD

10000 2 1% .12 0 1687 152 20000 30 8% .13 7.2 2850 1635 30000 82 23% .10 .7 2945 2114 40000 77 21% .09 .05 3640 2141 50000 60 17% .09 .05 4330 2929 60000 37 10% .05 3.8 3167 2161 70000 27 8% .07 .05 4737 3796 80000 18 5% .07 .06 5929 4797 90000 9 3% .05 .03 4769 3052

100000 6 2% .06 .05 6273 4584 110000 6 2% .07 .05 7663 5119 120000 2 1% .02 .03 4875 2438 130000 3 1% .05 .02 7225 2886 140000 0 0 .00 0 0 0 150000 0 0 .00 0 0 0 160000 1 <1% .03 0.0 4662 4662 aThe document word counts were rounded to the nearest ten thousand.

139

Descriptive Statistics for Published Articles Subpopulation (n=360)

Number of Referencesa Freq. Percent aDSI SD aDSW SD

50 62 17% .08 .07 2778 2420 100 116 32% .09 .05 3330 1927 150 93 26% .08 .06 3690 2512 200 51 14% .10 .07 4917 3265 250 18 5% .09 .06 5024 3228 300 9 3% .07 .05 4922 3660 350 7 2% .07 .05 5396 4439 400 1 <1% .11 .00 11917 0 450 2 1% .20 .09 17178 3977 500 0 0 .00 .00 0 0 550 0 0 .00 .00 0 0 600 1 <1% .07 .00 8872 0

aThe document references counts were rounded to the nearest fifty.

140

APPENDIX B

PUBLISHED ARTICLES FREQUENCY DETAIL TABLES

141

DSL Frequency Table for Published Article with Research Method Subpopulation (n=360)

Document Low Similarity Levels 0%-14% 15%-24% 25%-100%

Freq. % Freq. % Freq. %

Quantitative 91 31.9% 17 37.0% 14 48.3%

Qualitative 71 24.9% 10 21.7% 2 6.9%

Other 123 43.2% 19 41.3% 13 44.8%


Number of Wordsa Freq. Percent aDSI SD aDSW SD

10000 333 92% .12 .10 865 885 20000 27 8% .08 .10 1337 1546 aThe document word counts were rounded to the nearest ten thousand.


Number of Referencesa Freq. Percent aDSI SD aDSW SD

50 301 83% .11 .10 767 785 100 52 14% .12 .11 1435 1094 150 3 1% .20 .13 2684 1374 200 2 1% .25 .32 4148 5198 250 2 1% .05 .03 1160 602 aThe document references counts were rounded to the nearest fifty.

142

APPENDIX C

SPSS AND R SYNTAX

143

Download the CSV data, SAV data, SPSS syntax, and R Syntax files used in my study.

Zip File Download: Supporting CSV and SAV Data files, SPSS Syntax and R Syntax

CSV Files

• 360 Dissertation Corpus CSV file

• 360 Published Article Corpus CSV File

• 720 Combined Corpus CSV File

• 181 Substantive SSI Details CSV File

SPSS Data Files

• 360 Dissertation Corpus SAV file

• 360 Published Article Corpus SAV File

• 720 Combined Corpus SAV File

• 181 Substantive SSI Details SAV File

SPSS Syntax Files

• SPSS CSV Import to SAV Syntax SPS File

• SPSS COA Analysis SPS File

• SPSS Check Overs SPS File

• SPSS Charts and Graphs Syntax SPS File

R-Studio Syntax Files

• R-Studio Charts and Graphs Syntax R File

144

APPENDIX D

CSV FILES FIELD DESCRIPTIONS

145

Files: CSV_CORPUS-20171222.CSV CSV_DIS-20171222.CSV CSV_ART-20171222.CSV

dcmDOCNBR SRT System Generated Unique Document identifier

dcmDOCTYP 1=Dissertation 2=Published Article

dcmYEARMO Posted Year of Publication

dcmDOCCLS Document Class (Research Method=1,2,3, or 4)

dumQUANT Quantitative = 1 else 0

dumQUALT Qualitative = 1 else 0

dumMIXED Mixed = 1 else 0

dumOTHER Other = 1 else 0

dcmATHCNT Author Count

dcmPGECNT Page Count

dcmWRDCNT Word Count

wrkWRDCNT Word Count /10,000 and Rounded

dcmREFCNT Reference Count

wrkREFCNT Reference count /50 and Rounded

dcmGRSDSI Turnitin Reported DSI BEFORE any Adjustments or Exclusions

wrkGRSWRD Words based upon the Turnitin Reported DSI

dcmEXCDSI Turnitin Reported DSI AFTER any Adjustments or Exclusions

sumOTHCNT Number of POO Substantive SSI for this document

sumOTHAMT Average % for POO Substantive SSI for this document

sumOTHWRD Average similarity words for POO Substantive SSI for this document

sumSLFCNT Number of POS Substantive SSI for this document

sumSLFAMT Average % for POS Substantive SSI for this document

sumSLFWRD Average similarity words for POS Substantive SSI for this document

dcmMAXSSI Maximum SSI % found in this document

dcmFRQSSI Frequency of SSIs larger than 1% in this document

dcmSUMSSI Sum of SSI equal to and over 1% in this document

wrkADJDSI The Adjusted DSI (VERY IMPORTANT)

wrkADJWRD The adjusted Word similarities

wrkDIFDSI Difference between the dcmGRSDSI and the wrkADJDSI

wrkCOALVL The DSL category (1 = low, 2 = high, 3 = Excessive)

wrkLVLPCT Average group percent for the DSL in this Document's Level.

146

CSV_SSI-20170327.CSV

ssiDOCNBR SRT System Generated Unique Document identifier

ssiDOCTYP 1=Dissertation 2=Published Article

ssiDTLNBR SRT System Generated Detail Line Identifier when used with ssiDOCNBR

ssiSSITYP POO = Plagiarism-of-other POS = Plagiarism-of-self

ssiSSIAMT Substantive SSI Percentage of Similarity

ssiSSIWRD Substantive SSI Words that are Similar

147

APPENDIX E

TURNITIN COA REPORT (NO ADJUSTMENTS)

148

149

REFERENCES

Adhikari, N. (2010). Avoiding plagiarism and self-plagiarism. Journal of Nepal Pediatrics

Society, 30, 77-78.

Allen, G. N., Ball, N. L., & Smith, H. J. (2011). Information systems research behaviors:

What are the normative standards?. MIS Quarterly, 35(3), 533-A26.

Allison, P. (2013). What’s the best R-squared for logistic regression? Statistical

Horizons. Retrieved from http://statisticalhorizons.com/r2logistic

Amos, K. (2014). The ethics of scholarly publishing: Exploring differences in plagiarism

and duplicate publication across nations. Journal of the Medical Library

Association, 102(2), 87-91.

Anderson, J. A. (1984). On the existence of maximum likelihood estimates in logistic

regression models. Biometrika, 71(1), 1-10.

APA. (2010). Publication manual of the American Psychological Association (6th ed.).

American Psychological Association. Washington, DC: Author.

Auer, N. J., & Krupar, E. M. (2001). Mouse click plagiarism: The role of technology in

plagiarism and the librarian’s role in combating it. Library Trends, 49, 415-432.

Bailey, J. (2016). Why do students not understand plagiarism? Plagiarism Today.

Retrieved from https://www.plagiarismtoday.com/2016/09/01/why-dont-students-

understand-

plagiarism/?utm_source=Plagiarism+Today+Newsletter&utm_campaign=4bacb6f

aa7-RSS_EMAIL_CAMPAIGN&utm_medium=email&utm_term=0_643f84ace3-

4bacb6faa7-412694437

Baker, M. (2015). Smart software spots statistical errors in psychology papers.

150

Nature.com. Retrieved from http://www.nature.com/news/smart-software-spots-

statistical-errors-in-psychology-papers-1.18657

Barrett, R., & Malcolm, J. (2005). Embedding plagiarism education in the assessment

process. International Journal of Educational Integrity, 2(1), 38–45.

Batane, T. (2010). Turning to Turnitin to fight plagiarism among university students.

Educational Technology & Society, 13 (2), 1-12.

Bertolucci, J. (2013). Big data analytics: Descriptive vs. predictive vs. prescriptive.

Information Week. Retrieved from http://www.informationweek.com/big-data/big-

data-analytics/big-data-analytics-descriptive-vs-predictive-vs-prescriptive/d/d-

id/1113279?piddl_msgorder=thrd#msgs

Bedeian, A. G. (2014). "More than meets the eye": A guide to interpreting the

descriptive statistics and correlation matrices reported in management research.

Academy of Management Learning & Education, 13(1), 121-135.

doi:10.5465/amle.2013.0001

Bennett, K. K., Behrendt, L. S., & Boothby, J. L. (2011). Instructor perceptions of

plagiarism: Are we finding common ground?. Teaching of Psychology, 38, 29-35.

Berry, W. D., & Feldman, S. (1985). Multiple regression in practice. SAGE university

paper series on quantitative applications in the social sciences, 07-050. Beverly

Hills, CA. SAGE.

Bettencourt, L. A., & Houston, M. B. (2001). The impact of article method type and

subject area on article citations and reference diversity in JM, JMR, and JCR.

Marketing Letters, 12(4), 327-340.

Bewick, V., Cheek, L., & Ball, J. (2005). Statistics review 14: Logistic regression. Critical

151

Care, Vol 9 No 1. Retrieved from:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1065119/pdf/cc3045.pdf

Bham, G. H., Javvadi, B. S., & Manepalli, U. R. (2012). Multinomial logistic regression

model for single-vehicle and multivehicle collisions on urban U.S. highways in

Arkansas. Journal of Transportation Engineering, 138(6), 786-797.

doi:10.1061/(ASCE)TE.1943-5436.0000370

Braumoeller, B. F., & Gaines, B. J. (2001). Actions do speak louder than words:

Deterring plagiarism with the use of plagiarism detection software. Political

Science & Politics, 34(4), 835 – 839.

Bretag, T., & Mahmud, S. (2009b). A model for determining student plagiarism:

Electronic detection and academic judgment. Journal of University Teaching &

Learning Practice, 6(1).

Broome, M., Dougherty, M. C., Freda, M. C., Kearney, M. H., & Baggs, J. G. (2010).

Ethical concerns of nursing reviewers: An international survey. Nursing Ethics,

17, 741-748. doi:10.1177/0969733010379177

Burns, R. P. & Burns, R. (2009). Business Research Methods and Statistics Using

SPSS. Sage Publications Ltd., Los Angeles. CA.

Cabral-Cardoso, C. (2004). Ethical misconduct in the business school: A case of

plagiarism that turned bitter. Journal of Business Ethics, 49(1), 75-89.

doi:10.1023/B:BUSI.0000013864.76547.d5

Callahan, J.L. (2014). Creation of a moral panic? Self-plagiarism in the Academy.

Human Resource Development Review, Vol. 13(1) 3– 10.

doi:10.1177/1534484313519063

152

Chao, C., Wilhelm, W. J., & Neureuther, B. D. (2009). A study of electronic detection

and pedagogical approaches for reducing plagiarism. Delta Pi Epsilon Journal,

51, 31-42.

Cheema, Z., Mahmood, S., Mahmood, A., & Shah, M. (2011). Conceptual awareness of

research scholars about plagiarism at higher education level: Intellectual property

right and patent. International Journal of Academic Research, 3, 666-671.

Cheung, M., & Driver, D. (2004). Self-plagiarism as a social work concern. Hong Kong

Journal of Social Work, 38, 3-13.

Chronicle of Higher Education. (2002). Corruption plagues academe around the world.

Chronicle of Higher Education. Retrieved from

http://chronicle.com/article/Corruption-Plagues-Academe/16544

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale,

NJ: Erlbaum.

COPE. (2014). Promoting integrity in research publication. Committee on Publication

Ethics. Retrieved from http://publicationethics.org/cases

Cramer, D. (1998). Fundamental statistics for social research. London: Routledge.

Cramer, D., & Howitt, D. (2004). The SAGE dictionary of statistics. London: SAGE.

Creswell, J.W. (2014). Research Design (4th ed.). Sage Publications, Los Angeles. CA.

Cromwell, D. (2012). Punishing the pen with the sword?: Columbia's new, extreme, and

ineffective punishment for plagiarism. Pacific Rim Law & Policy Journal

Association.

David, W. (2011). A flap in Germany risks a storm in Spain. The Times, (United

Kingdom), 25.

Decullier, E., Huot, L., Samson, G., & Maisonneuve, H. (2013). Visibility of retractions: A

153

cross-sectional one-year study. BMC Research Notes, 6238. doi:10.1186/1756-

0500-6-238

Dee, T. S., & Jacob, B. A. (2012). Rational ignorance in education: A field experiment in

student plagiarism. Journal of Human Resources, 47, 397-434.

DeGeeter, M., Harris, K., Kehr, H., Ford, C., Lane, D., Nuzum, D., & Gibson, W. (2014).

Pharmacy students' ability to identify plagiarism after an educational intervention.

American Journal of Pharmaceutical Education, 78(2), 1-6.

EBSCO. (2016a). How is relevance ranking determined in EBSCO Discovery Service

(EDS)?. EBSCO Support. Retrieved from

http://support.epnet.com/knowledge_base/detail.php?id=3971

EBSCO. (2016b). Which fields can be searched using the "Select a Field" drop-down

list?. EBSCO Help. Retrieved from

https://help.ebsco.com/interfaces/EBSCO_Guides/General_Product_FAQs/fields

_searched_using_Select_a_Field_drop_down_list

Elbeck, M. (2009). Exploring the murky waters of self-plagiarism. Journal for

Advancement of Marketing Education, 14, 41-51.

ERIC. (2017). Online Submission Frequently Asked Questions. US Department of

Education. Retrieved from: https://eric.ed.gov/?submitfaq

Fahy, P. J. (2013). Uses of published research: An exploratory case study. International

Review of Research in Open and Distance Learning, 14(1), 145-166.

Fang, F. C., Steen, R., & Casadevall, A. (2012). Misconduct accounts for the majority of

retracted scientific publications. PNAS Proceedings of the National Academy of

Sciences of the United States of America, 109, 17028-17033.

154

doi:10.1073/pnas.1212247109

Farthing, M. G. (2006). Authors and publication practices. Science & Engineering

Ethics, 12, 41-52.

Field, A. (2011). Discovering Statistics using SPSS. Thousand Oaks, CA, US: SAGE

Publications Inc.

Fournier, G. (2016). Replicability. Psych Central. Retrieved from

https://psychcentral.com/encyclopedia/replicability/

Gibelman, M., & Gelman, S. R. (2003). Plagiarism in academia: Trends and

implications. Accountability in Research: Policies & Quality Assurance, 10(4),

229-252.

Gipp, B., Meuschke, N., & Breitinger, C. (2014). Citation-based plagiarism detection:

Practicability on a large-scale scientific corpus. Journal of The Association For

Information Science & Technology, 65(8), 1527-1540.

Grace-Martin, K. (2016). Outliers: To drop or not to drop. The Analysis Factor. Retrieved

from http://www.theanalysisfactor.com/outliers-to-drop-or-not-to-drop/

Grace-Martin, K. (2017). Explaining Logistic Regression Results to Non-Statistical

Audiences. The Analysis Factor. Retrieved from

http://www.theanalysisfactor.com/explaining-logistic-regression/

Grammarly. (2017). Plagiarism Detection. PremiumGrammary. Retrieved from:

https://app.grammarly.com/

Greenbird. (2009). The Urban Dictionary [blog post] Retrieved from

http://www.urbandictionary.com/define.php?term=Reverse+Plagiarism

Halupa, C. M. & Bolliger, D. U. (2013). Faculty perceptions of student self-plagiarism:

155

An exploratory multi-university study. Journal of Academic Ethics, 11, 297-310.

doi:10.1007/s10805-013-9195-6

Halupa, C. M. (2014). Exploring student self-plagiarism. International Journal of Higher

Education, 3(1), 121-126.

Handa, S. (2008, July). Plagiarism and publication ethics: Dos and don'ts. Indian

Journal of Dermatology, Venereology & Leprology. pp. 301-303.

Hatch, T., & Skipper, A. (2016). How much are PhD students publishing before

graduation?: An examination of four social science disciplines. Journal of

Scholarly Publishing, 47(2), 171-179.

Heather, J. (2010). Turnitin: Identifying and fixing a hole in current plagiarism detection

software. Assessment & Evaluation in Higher Education, 35(6), 1-15.

DOI:10.1080/02602938.2010.486471

Heckler, N. C., Rice, M., & Bryan, C. (2013). Turnitin systems: A deterrent to plagiarism

in college classrooms. Journal of Research on Technology In Education

(International Society For Technology In Education), 45(3), 229-248.

Hege, A. C. (2008). The effect of affective state on inadvertent plagiarism. University of

Virginia, ProQuest Dissertations Publishing, 3312170.

Hendee, W. R. (2007). A concern about plagiarism. Journal of Medical Physics, 32(4),

143-144.

Higher Education Commission, Pakistan. (n.d.). How to interpret originality report

(Guidelines). Retrieved from

http://www.hec.gov.pk/InsideHEC/Divisions/QALI/QADivision/Documents/Guideli

nes%20on%20Turnitin.pdf

156

Hill, J., & Page, E. (2009). An empirical research study of the efficacy of two plagiarism-

detection applications. Journal of Web Librarianship, 3(3), 169-181.

Hodges, J., & Lehmann, E. (1956). The efficiency of some nonparametric competitors of

the t test. Annals of Mathematical Statistics, 27, 324-335.

Honig, B., & Bedi, A. (2012). The fox in the hen house: A critical examination of

plagiarism among members of the Academy of Management. Academy Of

Management Learning & Education, 11(1), 101-123.

doi:10.5465/amle.2010.0084

Horner, J., & Minifie, F. D. (2011). Research ethics III: Publication practices and

authorship, conflicts of interest, and research misconduct. Journal of Speech,

Language, and Hearing Research, 54, S346-S362.

Howell, D.C. (2010). Statistical methods for psychology. Belmont, CA, US: Cengage

Wadsworth.

Hulten, Nicholls, Winslet, & Kmiot. (2000). The Committee on Publication Ethics

(COPE) Guidelines. Colorectal Disease, 2, 247-248.

Humphreys, L., & Klein, A. (2006). Measuring online social support: Can computer-

based text analysis approximate Burleson’s Person-Centered Hierarchy?.

Conference Papers -- International Communication Association, 1-29.

IDRE-UCLA. (2017A). Annotated SPSS output multinomial logistic regression. Institute

for Digital Research and Education. Retrieved from

http://www.ats.ucla.edu/stat/spss/output/mlogit.htm

IDRE-UCLA. (2017B). FAQ: What are pseudo R-squareds?. Institute for Digital

Research and Education. Retrieved from

157

http://www.ats.ucla.edu/stat/mult_pkg/faq/general/Psuedo_RSquareds.htm

iParadigms. (2011). Turnitin Instructor User Manual Chapter 2: Originality Check.

Turnitin Instructor Handbook, pp.48-60. Retrieved from

https://turnitin.com/static/resources/documentation/turnitin/training/Instructor_Ori

ginality_Report_Chapter_2.pdf

Ison, D. I. (2012). Plagiarism among dissertations: Prevalence at online institutions.

Journal of Academic Ethics, 10(3), 227-236. doi:10.1007/s10805-012-9165-4

Ison, D. I. (2014). Does the online environment promote plagiarism? A comparative

study of dissertations from brick-and-mortar versus online institutions. Journal of

Online Learning & Teaching, 10(2), 272-281.

iThenticate. (n.d.). 6 Consequences of Plagiarism. iThenticate. Retrieved from

http://www.ithenticate.com/resources/6-consequences-of-plagiarism

James, R., McInnis, C., & Devlin, M. (2002). Plagiarism detection software: How effective is

it?. Assessing Learning in Australian Universities. Retrieved from

http://www.cshe.unimelb.edu.au/assessinglearning/docs/PlagSoftware.pdf

Jent, H. C. (1967). Inverse plagiarism. Education Digest, 33(1), 32-34.

Jocoy, C., & DiBiase, D. (2006). Plagiarism by Adult Learners Online: A Case Study in

Detection and Remediation. International Review of Research In Open And

Distance Learning, 7(1), 1-15.

Jones, O. K. (2008). Practical issues for academics using the Turnitin plagiarism

detection software. Paper presented at the International Conference on

Computer Systems and Technologies, CompSysTech. O8, Bulgaria. Retrieved

from http://dl.acm.org/citation.cfm?id=1500935

Karabag, S., & Berggren, C. (2012). Retraction, dishonesty and plagiarism: Analysis of

158

a crucial issue for academic publishing, and the inadequate responses from

leading journals in economics and management disciplines. Journal of Applied

Economics & Business Research, 2, 172-183.

Karlsson, J., & Beaufils, P. (2013). Legitimate division of large data sets, salami slicing

and dual publication: Where does a fraud begin?. Knee Surgery, Sports

Traumatology, Arthroscopy, 21, 751-752.

Keck, C. (2006). The use of paraphrase in summary writing: A comparison of L1 and L2

writers. Journal of Second Language Writing, 15, 261–278.

Klienfield, Z. (2014). Web tool Turnitin could help UC Berkley combat plagiarism. The

Daily Californian. Retrieved from http://www.dailycal.org/2014/02/18/campus-

discusses-possible-introduction-web-tool-turnitin-help-bolster-honor-code/

Knight, M. (2013). Editing as reverse plagiarism. Monty Knight Counseling. Retrieved

from http://www.drmontyknightcounseling.com/2013/08/22/editing-as-reverse-

plagiarism/

Kock, N. (1999). A case of academic plagiarism. Communications of the ACM, 42, 96-

104.

Krejcie, R. V., & Morgan, D. W. (1970). Determining sample size for research

activities. Educational & Psychological Measurement, 30607-610.

Lee, K., Ahn, H., Moon, H., Kodell, R. L., & Chen, J. J. (2013). Multinomial logistic

regression ensembles. Journal of Biopharmaceutical Statistics, 23(3), 681-694.

doi:10.1080/10543406.2012.756500

Ling, S. (2006). Cultural backgrounds and textual appropriation. Language Awareness,

15(4), 264-282. doi:10.2167/la406.0

159

Liu, D. (2005). Plagiarism in ESOL students: Is cultural conditioning truly the major

culprit?. ELT Journal: English Language Teachers Journal, 59(3), 234-241.

doi:10.1093/elt/cci043

Marcus, S. & Beck, S. (2011). Faculty perceptions of plagiarism at Queensborough

Community College. Community & Junior College Libraries, 17, 63-73.

Marshall, T., Taylor, B., Hothersall, E., & Péérez-Martíín, L. (2011). Plagiarism: A case

study of quality improvement in a taught postgraduate programme. Medical

Teacher, 33(7), e375-e381. doi:10.3109/0142159X.2011.579201

Martin, D., Rao, A., & Sloan, L. (2011). Ethnicity, acculturation, and plagiarism: A

criterion study of unethical academic conduct. Human Organization. doi:

10.17730/humo.70.1.nl775v2u633678k6 Retrieved from

http://sfaajournals.net/doi/10.17730/humo.70.1.nl775v2u633678k6

Masic, I. (2012). Ethical aspects and dilemmas of preparing, writing and publishing of

the scientific papers in the biomedical journals. Acta Informatica Medica, 20, 141-

148. doi:10.5455/aim.2012.20.141-148

Massart, D.L., Smeyers-Verbeke, J., Capron, X., & Schlesierb, K. (2005). Visual

presentation of data by means of box plots. LC-GC Europe, 18(4) 215–218

Mawdsley, R. D. (2009). The tangled web of plagiarism litigation: Sorting out the legal

issues. B.Y.U. Education and Law Journal. 2009(2), 245-267. Retrieved from

http://digitalcommons.law.byu.edu/cgi/viewcontent.cgi?article=1259&context=elj

Mayes, R. (2016). Scholarly research tracker [computer software]. Available from

http://www.premierdatasoftware.com/srt.htm

Mayes, R. (2016). RandomSampler [computer software]. Available from

http://www.premierdatasoftware.com/srt.htm

160

http://www.robinjamesmayes.com/projects.html

McMurtry, K. (2001). E-cheating: Combating a 21st century challenge. T.H.E. Journal,

29(4), 36-38, 40-41.

Moore, D.S. & McCabe, G. P. (1989). Introduction to the practice of statistics. New

York, NY, US: W H Freeman/Times Books/ Henry Holt & Co.

Moore. (2008, April 2). Star terminates local columnist for plagiarism. Ventura County

Star (CA).

Moten, A. R. (2014). Academic dishonesty and misconduct: Curbing plagiarism in the

Muslim world. Intellectual Discourse, 22(2), 167-189.

MSU. (n.d.). Plagiarism. Michigan State University: Office of the University

Ombudsperson. Retrieved from https://www.msu.edu/~ombud/academic-

integrity/plagiarism-policy.html

Mulcahy, S., & Goodacre, C. (2004). Opening Pandora’s box of academic integrity:

Using plagiarism detection software. Proceedings from ASCILITE Conference

2004, Perth, WA.

Mullen, M. R., Milne, G. R., & Doney, P. M. (1995). An International marketing

application of outlier analysis for structural equations: A methodological note.

Journal of International Marketing, 3(1), 45-62.

Nefzger, M. D., & Drasgow, J. (1957). The needless assumption of normality in

Pearson's r. American Psychologist, 12(10), 623-625. doi:10.1037/h0048216

Neville, C.W. & Wadler, P. (2005). Beware the consequences of citing self-plagiarism.

Communications of the ACM, 48(6), 13.

O’Connell, A. A. & Rivet Amico, K. (2010). Logistic Regression. In G. R. Handcock &

161

R. O. Mueller (Eds.), The Reviewer’s Guide to Quantitative Methods in the Social

Sciences. Routledge, New York, NY.

O'Connor, S. J. (2010, May). What do duplicate publications, self-plagiarism and the

monotony of endless descriptive studies signify: Publication pressures or simply

a collective lack of imagination? European Journal of Cancer Care. pp. 281-283.

doi:10.1111/j.1365-2354.2010.01192.x

Office of Inspector General. (2014). National Science Foundation Office of Inspector

General semi-annual report to Congress. Retrieved from

http://nsf.gov/pubs/2014/oig14002/oig14002.pdf

Olive, M. L., & Franco, J. H. (2008). (Effect) size matters: And so does the calculation.

The Behavior Analyst Today, 9(1), 5-10. doi:10.1037/h0100642

Olt, M. R. (2007). A new design on plagiarism: Developing an instructional design model

to deter plagiarism in online courses. Capella University, ProQuest Dissertations

Publishing, 3277651.

Onwuegbuzie, A. J. & Daniel, L. G. (2005). Editorial: Evidence-based guidelines for

publishing articles in research in the schools and beyond. Research in the

Schools, 12(2), 1-11.

Onwuegbuzie, A. J., & Daniel, L. G. (2003). Typology of analytical and interpretational

errors in quantitative and qualitative educational research. Current Issues in

Education, 6(2). Retrieved from https://cie.ed.asu.edu/volume6/number2/.

Orim, S. I., Davies, J. W., Borg, E., & Glendinning, I. (2013). Exploring Nigerian

postgraduate students' experience of plagiarism: A phenomenographic case

study. International Journal for Educational Integrity, 9, 20-34.

162

Perfect, T. H., Defeldre A., Elliman, R. & Dehon, H. (2011). No evidence of age-related

increases in unconscious plagiarism during free recall. Memory, 19(5), 514-528.

Petrucci, C. J. (2009). A primer for social worker researchers on how to conduct a

multinomial logistic regression. Journal of Social Service Research, 35(2), 193-

205. doi:10.1080/01488370802678983

Posner, R. A. (2007). The little book of plagiarism. New York, NY: Pantheon Books.

PT. (2011). PlagAware takes top honors in plagiarism checker showdown. Plagiarism

Today. Retrieved from https://www.plagiarismtoday.com/2011/01/13/plagaware-

takes-top-honors-in-plagiarism-checker-showdown/

Randles, R. H., & Wolfe, D. A. (1979). Introduction to the theory of nonparametric

statistics. New York: Wiley.

Regmi, K., & Naidoo, J. (2013). Understanding the processes of writing papers

reflectively. Nurse Researcher, 20(6), 33-39

Retraction Watch. (2014). SAGE Publications busts “peer review and citation ring,” 60

papers retracted. Retraction Watch. Retrieved from

http://retractionwatch.com/2014/07/08/sage-publications-busts-peer-review-and-

citation-ring-60-papers-retracted/

Rhodes, M., Gelman, S. A., & Brickman, D. (2008). Developmental changes in the

consideration of sample diversity in inductive reasoning. Journal of Cognition &

Development, 9(1), 112-143. doi:10.1080/15248370701836626

Robinson, S. R. (2014). Self-plagiarism and unfortunate publication: An essay on

academic values. Studies in Higher Education, 39(2), 265-277.

Roig, M. (2010). Plagiarism and self-plagiarism: What every author should know.

163

Biochemia Medica, 20, 295-300

Rosenthal, R. (1991). Meta-analytic procedure for social research (2nd ed.). Newbury

Park, CA: Sage.

Rosnow, R. L., & Rosenthal, R. (2005). Beginning behavioral research: A conceptual

primer (5th ed.). Englewood Cliffs, NJ: Pearson/Prentis Hall.

Rouse, M. (2014). Document metadata. Content Management. Retrieved from

http://whatis.techtarget.com/definition/document-metadata

Rubio, C. C. (2013). Colombia’s poetic world of authors' moral rights: Considerations on

imprisoning a professor for plagiarism. Pacific Rim Law & Policy Journal, 22(1),

141-155.

Samuelson, P. (1994). Self-plagiarism or fair use. Communications of the ACM, 37(8),

21-25.

Schminke, M., & Ambrose, M. L. (2014). Retraction statement for 'Ethics and integrity of

the publishing process: Myths, facts, and a roadmap'. Management and

Organization Review, 10:1, March 2014, 157–162

Schwab, J. A. (2002). Multinomial logistic regression: Basic relationships and complete

problems. http://www.utexas.edu/courses/schwab/sw388r7/SolvingProblems/

Shanmugam, A. (2009). Citation practices amongst trainee teachers as reflected in their

project papers. Malaysian Journal of Library & Information Science, 14(2), 1-16.

Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality (complete

samples). Biometrika, 52(3/4), 591-611.

Shaw, V. N. (2002). Counseling the university professor on the securing of research

grants and the publishing of research products. Education, 123, 395.

164

Shelley, C. (2005). "Stolen words": A brief history and analysis of preaching and

plagiarism. Encounter, 66, 301-316.

Shi, L. (2006). Cultural backgrounds and textual appropriation. Language Awareness,

15(4), 264-282. doi:10.2167/la406.0

Siaputra, I. (2013). The 4PA of plagiarism: A psycho-academic profile of plagiarists.

International Journal for Educational Integrity, 9, 50-59.

Sibbald, B. (2000). Guest editorial: COPE guidelines on good publication practice: An

author’s view. Health and Social Care in the Community, 8(6), 355–361.

Siegel, E. (2016/12/07). Analytics + business expertise = actionable predictions for each

customer. Prediction Impact. Retrieved from

http://www.predictionimpact.com/customer-prediction.html

Simon, M. K. (2011). Dissertation and scholarly research: Recipes for success (2011

ed.). Seattle, WA, Dissertation Success, LLC.

Spielmans, G. I., Biehn, T. L., & Sawrey, D. L. (2010). A case study of salami slicing:

Pooled analyses of duloxetine for depression. Psychotherapy and

Psychosomatics, 79(2), 97-106. doi:10.1159/000270917

Stevens, J. P. (2009). Applied multivariate statistics for the social sciences 5th Ed. New

York, NY: Routledge.

Sun, Y. (2013). Do journal authors plagiarize? Using plagiarism detection software to

uncover matching text across disciplines. Journal of English for Academic

Purposes, 12(4), 264-272. doi:10.1016/j.jeap.2013.07.002

Susser, M. & Yankauer, A. P. (1993). Prior duplicate, repetitive, fragmented, and

redundant publication and editorial decisions. American Journal of Public Health,

165

83, 792–794.

Sutherland-Smith,W. & Carr, R. (2005). Turnitin.com: teachers’ perspectives of anti-

plagiarism software in raising issues of educational integrity. Journal of University

Teaching and Learning Practice, 2(3b), 94–101

Swanson, R. A. (1995) Human resource development: Performance is the key. Human

Resource Development Quarterly, 6(2), 207 – 13.

Technavio, R. (2016). Top 8 vendors in the anti-plagiarism software market for

education from 2016 to 2020: Technavio. Business Wire (English). Retrieved

from http://www.businesswire.com/news/home/20161011005155/en/Top-8-

Vendors-Anti-Plagiarism-Software-Market-Education

Thomas, A., & de Bruin, G. P. (2014). Plagiarism in South African management

journals. South African Journal of Science, 111(1/2), 1-3.

doi:10.17159/sajs.2015/20140017

Thumwimon, S., & Takahashi, Y. (2010). A prospective process for implementing

human resource development (HRD) for corporate social responsibility (CSR).

Interdisciplinary Journal of Contemporary Research in Business, 2(1), 10-32.

Toulouse Graduate School. (2016). Preparation and filing of dissertations and theses

(Spring 2016 Revision). University of North Texas. Retrieved from

http://tsgs.unt.edu/thesis-manual

Tucci, V., & Galwankar, S. (2011). JETS policy on plagiarism and academic dishonesty.

Journal of Emergencies, Trauma & Shock, 4, 3-6. doi:10.4103/0974-2700.76818

Turmfalke, (2010). The Urban Dictionary. Retrieved from

http://www.urbandictionary.com/define.php?term=Reverse+Plagiarism

166

Turnitin. (2011). Plagiarism and the web: Myths and realities: White paper. iParadigms.

Retrieved from http://pages.turnitin.com/PlagiarismandtheWebHE.html

UMUC. (2016). Reading and Understanding Turnitin Originality Reports. Retrieved from

http://www.umuc.edu/library/libhow/turnitinoriginality_tutorial.cfm

U.S. Government. (2014). Inspector General Act of 1978, as amended. Retrieved from

http://www.ignet.gov/pande/leg/igactasof1010.pdf

Wager, E. & Williams, P. (2011). Why and how do journals retract articles? An analysis

of Medline retractions 1988-2008. Journal of Medical Ethics. 37, 567-570.

doi:10.1136/jme.2010.040964

Walker, J. (2010). Measuring plagiarism: researching what students do, not what they

say they do. Studies in Higher Education, 35(1), 41–59.

Weisgrau, S. L. (2011). Talking about brevity with high school seniors. Vocabula

Review, 13(7), 1-4.

White, R.T. & Arzi, H.J. (2005). Longitudinal studies: Designs, validity, practicality, and

value. Research in Science Education, 35, 137-149.

Willis, M. (2016). Why do peer reviewers decline to review manuscripts? A study of

reviewer invitation responses. Learned Publishing, DOI: 10.1002/leap.1006

Wilmington, T. (2013, November 21). Acrobat Reader Discussions, Adobe

Communities. Retrieved from

https://forums.adobe.com/thread/1110154?start=0&tstart=0

Wittmaack, K. (2005). Penalties plus high-quality review to fight plagiarism. Nature, 436,

24. doi:10.1038/436024d

Yank, V. & Barnes, D. (2003). Consensus and contention regarding redundant

167

publications in clinical research: Cross-sectional survey of editors and authors.

Journal of Medical Ethics, 29, 109–114.

Yentis, S. M. (2010). Another kind of ethics: From corrections to retractions editorial.

Anaesthesia. pp. 1163-1166. doi:10.1111/j.1365-2044.2010.06557.x

Youmans, R. J. (2011). Does the adoption of plagiarism-detection software in higher

education reduce plagiarism? Studies in Higher Education, 36, 749-761.

doi:10.1080/03075079.2010.523457

Zhang, Y., & Jia, X. (2012). A survey on the use of CrossCheck for detecting plagiarism

in journal articles. Learned Publishing, 25, 292-307. doi:10.1087/20120408

Zimmerman, D. W. (1996). An efficient alternative to the Wilcoxon signed-ranks test for

paired nonnormal data. Journal of General Psychology, 123(1), 29.