Perceptions of Education Research 1 Running head: GOOD ...kdbrad2/MWERA_GoodResearch.pdfPerceptions of Education Research * Working Draft * Do not cite without permission of contact

Perceptions of Education Research

*** Working Draft *** Do not cite without permission of contact author.

1

Running head: GOOD EDUCATION RESEARCH

A consideration of ethics, methods and theory:

Is there consensus on what constitutes good education research?

Kelly. D. Bradley∗, Kenneth D. Royal, Jessica D. Cunningham, Jennifer A. Weber,

Jennifer A. Eli, Tara Baas, and William E. Harris, Jr.

University of Kentucky

Special Thanks to Eleanora Byrd, University of Kentucky, for survey management

∗ Use Kelly D. Bradley, Ph.D. as contact: 131 Taylor Education Building, Lexington, KY 40506; [email protected]



2

Abstract

The question of what constitutes good education research has received a great deal of attention.

Even though as a whole, purpose, theme and methods vary across research agenda, it seems that

the education community could have some consensus on what constitutes good research. For the

purpose of this study, a survey investigating faculty and graduate student perceptions of good

education research was administered in the College of Education at a large Southeastern

university. The primary goal of the study was to reveal which characteristics are most frequently

endorsed by members of various disciplines within education research. Results suggest that items

connected to ethics and theory were empirically found to be relatively easy to endorse for the

majority of participants. When connected to methodology, variation in perceptions was

noticeable.



3

A consideration of ethics, methods and theory:

Is there consensus on what constitutes good education research?

Faculty and graduate students are inundated with and trained to be well-versed in

research methods, theory, ethics and other common elements associated with the research

process. In both the classroom and the profession, numerous ways to conduct research exist.

Some education researchers prefer employment of quantitative methods, qualitative methods, or

a combination of the two approaches. Regardless of the approach and form of inquiry, it is

reasonable to assume that common ground and variation in the perceptions of what constitutes

good education research can be identified.

Connecting to the general theme of standards in conducting research in education, this

paper sets out to explore the question of what constitutes good education research. Inspired by an

article published in the Educational Researcher (Hostetler, 2005), this discussion is centered on

the results of a survey about perceptions of good research completed by College of Education

(COE) faculty and graduate students at a public southeastern university. Using the American

Educational Research Association (AERA) and the American Psychological Association (APA)

as the guiding sources, statements about good research were derived from the organizations’

guidelines and developed into survey items. The results of the survey along with the existing

literature base will serve as a foundation for operationalizing good education research.

As the survey instrument is in the early stages of construction, a Rasch measurement

model was employed to evaluate the quality of the survey instrument in terms of rating scale

appropriateness and clear divisions between established constructs. A hierarchy of survey items

is presented to illustrate endorsability of characteristics of good research. The findings are



4

discussed in three overarching areas: ethics, methods and theory. The paper stimulates dialogue

about these areas and how each functions as a driving force for education research.

Theoretical Framework

AERA and APA were chosen as the guiding authorities on the topic of good education

research as both govern and direct research and practice in the domains instructed throughout the

college of education. To better inform and guide the research, an overview of the literature as it

pertains to the three major domains ethics, methods and theory is presented, specifically

literature relating to perceptions of good research. Also, a brief overview of the Rasch model is

presented to inform the methods and results which follow.

Ethical Considerations

Hostetler (2005) argues that recent debates regarding good research and its relationship to

ethics can be limited to methodology. The bulk of literature regarding ethics and good research

seem to support this assertion. Social science researchers tend to agree about what is and is not

proper when conducting scientific inquiry (Babbie, 2008). Several ethical considerations have

been highlighted in the literature and are investigated by this study.

Numerous ethical considerations involve participation, consent, and dissemination of

results. Concerning participation, research volunteers must participate in studies on their own

free will (Nardi, 2003) without being forced or otherwise coerced (Babbie, 2008; Simmerling,

Schwegler, Sieber, & Lindgren, 2007). Participants should also be granted confidentiality to

prevent the linking of personal identity to various responses; and unless stated otherwise,

participants should remain anonymous as well (AERA, 2002). Further, participants should

always be granted informed consent. That is, participants should be made aware of their rights

and informed of any risks or possible repercussions that could result from their participation in



5

the study. Typically, Institutional Review Boards (IRB) monitor related research ethics to ensure

appropriate measures are taken to protect potential participants. Similarly, a number of issues

relating to the dissemination of research findings are discussed in the literature.

According to Babbie (2008), researchers are ethically obligated to admit unexpected

findings and the limitations of their work. The AERA encourages researchers to report their

findings without selectivity and/or secrecy. That is, researchers should objectively report results

and refrain from distributing only the results that are deemed favorable to their personal beliefs.

Likewise, researchers are obligated to report any results that may appear unfavorable. Although

the language used varies slightly, the literature regarding ethics and methodology is generally

grounded in these ethical guidelines. Beyond the topic of ethical guidelines and methodology,

the literature begins to diverge into small segments regarding the ethical context of research

concerns.

The literature addresses the changing role of researchers utilizing qualitative

methodology, including researcher obligations to examine the ethics and morality of their work

and to explicitly acknowledge the fragility of the relationship between the researcher and those

being researched (Ebbs, 1996). In a similar vein, the researcher/researched relationship has

become controversial in the context of “outsider research.” The argument centers on the

epistemological claim that researchers who are not members of disempowered groups should not

attempt to study these groups. The outside researcher cannot accurately understand or represent

such groups and faces an ethical dilemma of exploiting or being disrespectful of participants

(Bridges, 2001).

The literature also addresses issues pertaining to researchers’ expectations and ethical

standards. Professional ethics in academe can be viewed as cultural, personal, and academic, but



6

are often evaluated situationally, thus making it difficult to find an extensive yet accessible code

of conduct for academics (Bruhn, Zajac, Al-Kazemi, & Prescott, Jr., 2002). Another dilemma

facing researchers is the origin of ethical guidelines for researchers, The Belmont Report.

However, the report’s emphasis on medical issues does not seamlessly transfer into the social

and behavioral science environments (Simmerling, et al, 2007).

Ethical considerations in research writing are often internalized, thus the visibility of

these components of research can be masked in publications. Authors are limited by criteria

imposed by journal editors regarding structure and content. The evidentiary condition of

arguments in research literature can be viewed as impeding writers’ prerogatives to address

ethical implications or considerations of their work in manuscripts. The nature of research

articles may not allow the moral dimension of research to be explicitly noted in journals

(Kansanen, 2003). Discussions pertaining to the implicit nature of ethics in research have led

researchers to address a need for improved ethics education. Researchers have advocated that

scientific societies offer ethics education programs to members and graduate students (Iutcovich,

Kennedy, & Levine, 2003), and have argued for more comprehensive ethics training in classes

(Muskavitch, 2005).

Although the generally recognized guidelines have been presented, work connecting the

quandary of research and ethics to the need for improved ethics education is not apparent. A

body of literature related to how researchers navigate ethical dilemmas, as well as how they

conceive the role of ethics in research beyond methodology, would be beneficial in making the

argument that stronger ethics education would (or would not) lead to better research practice.

Methodological Considerations



7

Educational research is “the collection and analysis of information on the world of

education so as to understand and explain it better” (Opie, 2004). Thus, “high-quality” education

research would not only involve using carefully constructed designs, but also implementing the

most appropriate methods of data collection and analysis with the purpose of answering research

questions aimed at understanding the world of education. More often than not, “high-quality”

research concludes as debate on a methodological level. Methodological debates in education

research and thoughts on the nature of knowledge and learning play a vital role in how one

determines what is or is not “high-quality” education research (Hostetler, 2005).

For the purposes of this paper we make a distinction between methods and methodology.

Methods are the techniques and/or procedures for data collection and analysis. A methodology

refers to a philosophical framework, strategy, or plan of action implemented with the intention of

obtaining knowledge by considering the best methods or procedures leading to data that will

provide evidentiary basis for what is being studied (Creswell, 2003; Creswell & Plano-Clark,

2007; Sikes, 2004). “Methodology is concerned with the description and analysis of research

methods rather than the actual, practical use of those methods” (Sikes, 2004, p. 16). For example,

the methodology for this project is survey research as it is a “strategy or plan of action that links

methods to outcomes governs our choice and use of methods” (Creswell, 2003, p.5). The method

used for data collection is a Web-based survey and the method for data analysis involves

implementation of the Rasch model.

Two major factors influence a researcher’s choice of procedures and methodology when

conducting educational research: 1) his/her philosophical assumptions about reality, knowledge

and values; and 2) the research question(s) under investigation. Educational researchers

inherently bring to the table certain philosophical assumptions that guide their approach to



8

conducting educational research (Creswell, 2007). In particular, their views on the nature of

reality (ontology), the nature of knowledge (epistemology), and the nature of human values

(axiology) play a crucial role in how one carries out research (Sikes, 2004). Researchers might

ask themselves the following questions to ascertain underlying philosophical assumptions that

guide their approach to research: Is there a single reality or multiple constructed realities? Is the

relationship between the researcher and the subject independent or dependent? Is inquiry value

free or value bounded? Tashakkori and Teddlie (1998) outline four philosophical orientations

that represent the major stages of the paradigm wars prior to World War II and up to the 1990s:

logical positivism, post positivism, constructivism, and pragmatism. Consider these

philosophical orientations along a continuum with logical positivism on the far left and

constructivism on the far right. Logical positivists believe in a single reality; the relationship

between researcher and subject is independent; knowledge is objective; inquiry is value free,

context-free generalizations are possible; and causal links can be isolated and identified.

Quantitative research methods are most often identified with the logical positivism philosophical

orientation as these methods propose measuring and analyzing causal relationships within a

framework that is value-free. Quantitative research methods generally involve working with

numerical data collected from a representative sample and analyzed using statistical methods.

Likewise, with qualitative research, the researcher and the subject are seen as independent;

knowledge is viewed as objective; and methods use deductive reasoning.

Post positivism became popular after World War II. Unlike logical positivists, post

positivists believe that reality is constructed; knowledge is objective; inquiry is influenced by the

values of the researcher; context-free generalizations are possible; causal links can be isolated



9

and identified; and strive for separation between researcher and subject while acknowledging

their own biases

Moving into the 1960s, the emergence of constructivism can be seen as lying to the far

right of the continuum. Constructivists believe in multiple constructed realities; the relationship

between researcher and subject is dependent; knowledge is negotiated; inquiry is value-bounded;

and context-free generalizations are not possible. Qualitative research methods are most often

identified with the constructivist philosophical orientation as these methods involve analyzing

descriptive data using inductive reasoning to draw inferences.

The fourth major stage, pragmatism, focuses on practicality emphasizing a “what works”

best approach. Pragmatists believe in singular and multiple realities (Creswell, 2007).

Pragmatists take “multiple stances” when it comes to axiology as they would “include both

biased and unbiased perspectives” (p.24). The pragmatist paradigm allows for the use of mixed

methods in educational research, that is, researchers can “combine the qualitative and

quantitative approaches into the research methodology of a single study or multiphase study”

(Tashakkori & Teddlie, 1998, p. 18).

To summarize, quantitative and qualitative approaches to educational research have been

distinguished and thereby defined on the basis of the type of data used (numeric or descriptive),

the mode of analysis (statistical or interpretative), the logic employed (deductive or inductive),

the type of investigation conducted (explanatory or exploratory), focus of the research (quality or

quantity), as well as on the basis of underlying paradigms (positivism or constructivism)

(Tashakkori & Teddlie, 1998; Merriam, 1998; Creswell & Plano-Clark, 2007). A combination of

qualitative and quantitative methods is regarded as mixed methods. As argued in the literature:



10

A mixed methods study involves the collection or analysis of both quantitative and/or

qualitative data in a single study, in which the data are collected concurrently or

sequentially, are given a priority, and involve the integration of the data at one or more

stages in the research process (Creswell, Plano-Clark, Gutmann, & Hanson, 2002, p.

212).

The second major factor influencing a researcher’s procedures and methodology is the

research question(s). “The important issue is to ensure that the research procedure(s) used is

appropriate to the research question being asked and the research answers being sought” (Opie,

2004, p.9). “Research methods should follow research questions in a way that offers the best

chance to obtain useful answers” (Johnson & Onwuegbuzie, 2004, p.18). Once a researcher has

selected a methodology, one must be able to justify or provide a rationale for choosing a

particular approach or procedure that he or she feels will best address the research question. “It is

on the match between methodology and procedures and research focus/topic/questions that the

credibility of any findings, conclusions and claims depends, so the importance of getting it right

cannot be overemphasized” (Sikes, 2004, p. 17).

Theoretical Considerations

Kerse and Elley (2003) believe defining good research is as subjective as defining beauty,

as beauty lies in the eyes of the beholder. Brandon (2000) has simply defined “good” research as

any project producing useful results whether positive or negative. Interestingly, Brandon posits,

“Research is frequently frustrating and often leaves us more confused than when we started” (p.

1). In multiple texts and articles, many experts in the field of research have referred to research

as a “science”. Whitley (1996) describes science as the “systematic process for generating

knowledge about the world… consisting of three important aspects; the goals of science, key



11

values of science, and perspectives on the best way in which science can go about generating

knowledge”(p. 2). Making this reference presumes a great deal of meticulous efforts take place

when research is performed. Additionally, scholars could speculate about the driving force

behind quality research as a science.

One factor which many consider essential to “high quality” research is theory. Multiple

researchers and educators (Brandon, 2000; Elmes, Kantowitz, & Roediger, 1995; Hughes, 1999;

Kerse & Elley, 2003; Mikk, 2006; Nardi, 2003; Whitley, 1996) have noted the importance of

theory in quality research. Hughes (1999) attested that every piece of research should have

definite connections to some theory or existing body of literature. The most common strategy of

connecting current research to theory is through a literature review (Nardi, 2003). Nardi stated an

in depth review of literature provides an introduction of theories, which leads to a formulation of

questions and concepts used to test current theories or create new ones.

Scholars have often labeled theory as the building block of quality research (Brandon,

2000; Hughes, 1999; Mikk, 2006; Nardi, 2003). Hughes (1999) believes without links to

previous theory, researchers become suspect about the possibility of repeating earlier studies.

When faculty, students, and private investigators begin the early stages of their research, they

rely upon existing theories to help guide their studies (Brandon, 2000). According to Mikk

(2006), theory allows the researcher to establish a hypothesis in order to answer the research

question. So, how do researchers define theory?

According to Nardi (2003), theory is defined as “a set of statements logically linked to

explain some phenomena in the world around us” (p. 27). Whitley (1996) defines theory as “a

set of statements about relationships between variables” (p. 9). He continues to characterize

variables as abstract constructs (e.g., memory) or concrete objects (e.g., length). Some



12

researchers have alluded to judging the relevance of a theory (Elmes, Kantowitz, & Roediger,

1995) by the larger number of occurrences or observations of a theory or the fewer number of

statements of a theory implying a better theory. But, this is not always true in every situation.

Elmes, Kantowitz, and Roediger (1995) and Whitley (1996) have discussed the two

major functions of theory in research practice as organization and prediction. First, theory

provides a framework for the “systematic and orderly” (Whitley, 1996, p.7) presentation of data.

Second, theory allows researchers to introduce predictions for occurrences for which no data is

collected. Whitley also introduced a third major function of theory in research practice – as

theory organizes data, it also extends the current knowledge in the field. In other words, data

from theory-driven research can modify current theories, establish linkage to other theories,

allow the merging of theory through a convergence of concepts, and extend knowledge across

disciplines.

Many researchers have acknowledged the important relationship between theory and

research (Nardi, 2003; Whitley, 1996). This relationship has fashioned two forms of rationale -

deductive and inductive reasoning. Deductive reasoning utilizes hypotheses from a theory to test

that theory. On the other hand, inductive reasoning uses data from previous tests of theory to

verify or modify certain aspects of the theory when results fail to provide support. Nardi states:

“Most research involves both processes: a review of literature tells us what theories and

explanations we can use to deduce specific research questions that are then used to get

data that form the basis for inducing or modifying a theoretical perspective to explain

what was observed and measured” (p. 28).

In an attempt to summarize theoretical considerations, theory has been defined as the

building block to quality research. Researchers rely on theory to help organize data and to



13

formulate new hypotheses for their current research. Further, researchers often link previous

theories with current theories, or in some cases, create entirely new theories. Observing the

importance of theory in quality research has created theory-based practices, instrument

development, and other applications relevant to educators and researchers. One must not forget

the importance of theory in the formation of research hypotheses. With this in mind, the

expectation for high-quality research is to be theoretically grounded or linked to previous

research.

An introduction to the Rasch model

The Rasch model is a one-parameter logistic model within item response theory (IRT) in

which the amount of a given latent trait in a person and the amount of that same latent trait

reflected in various items can be estimated independently yet still compared explicitly to one

another. Given the Rasch model follows mathematically from the requirement of invariance of

comparisons among persons and items, a Rasch analysis is appropriate when the total score on a

questionnaire is used to make inferences about an individual’s level of a latent trait inherent in

that individual. Classical Test Theory (CTT) also uses the total score to characterize each person,

but it asserts the total score as the relevant statistic with little consideration of anomalies in the

items or the respondents. Applications of the Rasch model account for these anomalies and

provide a more informative score (Andrich & Luo, 2003).

The Rasch model, introduced by Georg Rasch (1960), yields a comprehensive picture of

the construct under measurement and the respondents on that measure. It allows observations of

respondents and items to be connected in a way that indicates the occurrence of a certain

response as probability rather than certainty and maintains order in that the probability of

providing a certain response defines an order of respondents and items (Wright & Masters,



14

1982). The model is static, meaning that for each person having a certain amount of the given

latent trait; the model specifies the probability of a response in one of the categories of an item.

The response process is a classification into ordered categories defined by thresholds, the point

where the probability of a response in either one of two adjacent categories is 50%, analogous to

the markings on a ruler (Andrich & Luo, 2003; Guttman, 1950).

The Rasch model uses the sum of the item ratings simply as a starting point for

estimating probabilities of those responding to each item. In the case of a questionnaire,

probabilities are based upon individuals’ willingness to endorse a set of items and the difficulty

to endorse those items. The difficulty to endorse is assumed to be the main characteristic

influencing responses (Linacre, 1999). Rasch analysis reports both person ability (i.e.,

willingness-to-endorse) and item difficulty (i.e., difficulty-to-endorse) estimates along one logit

(log odds unit) scale, “a unit interval scale in which the unit intervals between the locations on

the [combined person-item scale] have a consistent value or meaning” (Bond & Fox, 2001, p.

29). Bond and Fox explain that employing Rasch techniques allows for the ordering of

respondents along this continuum of ability or willingness to endorse items, and orders items

along a continuum of difficulty to endorse. Rasch measurement is relevant whenever a

questionnaire or assessment is constructed to measure the degree of some property inherent in

persons or other entities, as is the case in this study.

Purpose

The purpose of the study was both exploratory and evaluative in nature. Data collected

via the Web-based survey are intended to help investigate characteristics that faculty and

graduate students in the college of education associate with good education research as well as

evaluate the use of the online survey. The primary expectation of the study was to reveal which



15

characteristics are most frequently endorsed by members of various disciplines within

educational research. The secondary expectation was to find support for the quality of the Web-

based measure employed. Initial data collected in the pilot study using the Web-based

measurement tool provide guidance for revisions of the instrument, which will undergo similar

subsequent analysis to assess the validity of the revised survey. Specifically, the objectives of the

study were to: 1) illustrate an overall picture of participants’ characterization of “good” or

“quality” research; 2) examine differences in preferences based on participant demographics, and

3) validate the Web-based survey to be used for data collection.

Research Questions

The data collected in the study were used to help answer questions such as:

1) What do graduate students and faculty consider to be “good” or “quality” educational

research?

2) What differences exist in participants’ endorsement of “quality” research characteristics

based on participant demographics?

3) Is the survey a valid and quality method of data collection regarding student and faculty

impressions of educational research?

Methods

Response Frame

Two primary populations have been targeted for this study. The sampling frame includes

faculty (85) and graduate students (approximately 800) from the College of Education (COE) at

a public southeastern university. The university’s COE includes undergraduate- and graduate-

level programs in six areas of educational research and practice, including: curriculum and

instruction (i.e.: teaching and learning); educational leadership studies; educational and



16

counseling psychology; educational policy and evaluation; kinesiology and health promotion;

and special education & rehabilitation counseling. All respondents were expected to be at least

21 years of age. No criteria were provided for exclusion, as all COE faculty and graduate

students were surveyed. The foreseen demographic profile anticipated for the study participants

was conceptualized based on 2006 Headcount Enrollment data provided by the university’s

Office of Institutional Research, which expected the student sample should consist of

approximately 72% female and 28% male; 77% White/Caucasian, 10% African American, and

the remainder of the sample consisting mostly of Asian, Hispanic, and nonresident alien students.

Procedure

A formal request to the Registrar’s office was sent requesting all College of Education

faculty and graduate students’ email addresses. Concurrently, an email notice was sent via COE

departmental LISTSERVs to inform potential respondents that the survey should arrive in their

email inbox in the near future. The sampling frame received an email invitation to participate in

the survey, which was embedded with the link to the survey. The email message also contained

statements regarding the following: purpose of the survey; informed consent; voluntary nature of

the survey ensuring no penalties or repercussions of any kind regardless of whether or not a

person chooses to participate; confidentiality; aggregation of data; amount of time anticipated to

complete the survey; timeline of when the survey will be closed; and contact information should

potential respondents have any questions and/or concerns.

Responses were collected and stored on a secure web server provided by the Web-based

survey software program SurveyMonkey (2007), which is hosted on secure servers. Once the

survey was closed, data were downloaded onto the primary investigator’s computer. From there,

data were burned onto a CD and stored in a locked file cabinet in the principle investigator’s



17

office. With regard to data analysis and dissemination of results, no personally identifiable data

are presented. All data appear in aggregate form.

Instrumentation

The survey instrument consisted of 60 questions partitioned into two components, namely

statements describing potential characteristics of good education research (39) followed by a

demographics component (21). Statements of good education research were grouped into three

domains: methodological (17), ethical (10), and theoretical (12). All survey questions were

written or adapted from guidelines for “quality” or “good” research provided by various research

organizations which lead and govern research in the given educational disciplines. Namely,

questions resemble guidelines set forth by the American Educational Research Association

(AERA), American Psychological Association (APA), and National Council of Teachers of

Mathematics (NCTM). The demographic component of the instrument included items which

encompass both the individual and professional background of the participant. These data were

designed to assist in determining if any differences exist in participants’ endorsement of

“quality” research based on various demographic criteria. All questions were devised, reviewed

and revised by the research team developing the survey.

Analysis

Crucial to determining common characteristics of ‘good’ education research is the quality

of the instrument used to examine participant responses. Bond and Fox (2001) argue,

“…interpretation of analyses can only be as good as the quality of the measures” (p. 26). Prior to

the full scale administration of the survey instrument, the quality of the instrument must be

assessed. In an effort to ensure measurement stability, this study employs a one-parameter Item

Response Theory model, otherwise known as the Rasch model.



18

An overall partial credit model was applied with persons and items to test the overall fit

of the data to the model using Winsteps version 3.57.4 software. The mathematical expression

employed for constructing measures through responses is log (P nik / P )1( −kni )/B n –D ik (Andrich,

1978) where P nik represents the probability the person n when responding to item i would be

observed in category k; similarly P )1( −kni ; B n is the attitude of person n; D ik is the difficult of

item i with the impediment to being observed in category k relative to k-1. The partial credit

model, mathematically equivalent to the rating scale model within Winsteps, was chosen due to

the possibility that respondents may interpret the scale differently depending on the item.

Missing data were treated as missing since it was reasonable to believe respondents might not

wish to answer all survey items with integrity. Further, it was decided that imputing means or

other substitutes for missing data would be inappropriate since missing data are not problematic

with the Rasch model and could result in data and information not reflective of the answers

provided by the respondents.

First, the Rasch model was employed to guide revisions to the survey instrument prior to

the full scale study. Survey items and respondents not adequately fitting the model requirements

were identified using the mean square fit statistics, with a reasonable range determined within

one standard deviation of the average mean square fit statistic (Wright & Stone, 2004).

Inspecting the outfit mean-squares provides evidence about the fit of the data to the model. The

infit mean-squares are used to determine the fit of the item within the construct. The guidelines

outlined by Linacre (2004) were used to evaluate the rating scale category effectiveness based on

the responses. Point-biserial correlations were inspected to investigate the orientation of the

latent variable to ensure that the polarity of the items were of the same sign, or direction (i.e. all

point-biserial correlations were positive). The number of observations and the distribution of



19

observations across categories were examined to describe the functioning of the rating scale

categories. Probability curves were used to visually inspect the rating scale category function.

Advancing average measures with each category and step calibrations ensure the rating scale

measure is stable and accurate.

Finally, Rasch results and descriptive statistics were used to present findings from the

study. Relative frequency tables illustrated a general picture of the characteristics the participants

associate with good education research. The item map produced from the Rasch analysis

provided an empirical hierarchy, which represents characteristics of good research that are most

likely to receive endorsement.

Results

Demographic information for participants

The sample for this study consisted of 126 faculty (n = 50; 39.7%) and graduate students

(n = 76; 60.3%) from UK’s College of Education. Below are personal demographic

characteristics for both faculty and graduate students. First, Table 1 examines faculty

demographic characteristics:

[Insert Table 1 Here] Graduate students also comprise a significant portion of this sample. Of the 76

respondents, 50 (69.4%) reported full-time status, 14 (19.4%) part-time status, and 8 (11.1%)

inactive. Table 2 examines graduate student demographic characteristics:

[Insert Table 2 Here] With regard to the collective sample, 89% (n = 105) of respondents reported being of

White/Caucasian ethnicity. Asian and Pacific Islanders comprised 5.9% and African American



20

or Black respondents accounted for 5.1%. A plethora of areas of academic interest were reported

from the sample. The results are presented in Table 3 below.

[Insert Table 3 Here]

Respondents were asked several questions to further detail their preferences regarding

research (see Table 4). When given the forced-choice question to indicate a primary interest in

either teaching or research, 43.4% of faculty and students reported “very heavily” or “leaning

toward” research. Respondents were also asked to estimate the number of hours spent on

research in a given week. The majority of respondents (46.0%) reported spending one to nine

hours on research each week, followed by approximately one fourth (24.2%) spending 10-19

hours each week on research. Faculty and students were also asked to report their primary

research methodology, where 40.7% reported mostly or entirely quantitative methods. When

asked about their perceptions of educational research quality, the majority (57.5%) reported

average quality. For complete results, see Table 4 below.


Fit of data to the model

First, person and item reliability estimates are important in determining fit of the data to

the model. With a total of 137 participants measured, the person reliability and separation are

0.86 and 2.48, respectively. For 39 survey items, the item reliability and separation estimates

were 0.98 and 7.64, respectively. Point-biserial correlations were inspected to investigate the

orientation of the latent variable to ensure that the polarity of the items were of the direction (i.e.

all point-biserial correlations were positive), which was the case.

Survey items and respondents not adequately fitting the model requirements were

identified using the mean square fit statistics, with a reasonable range determined within one



21

standard deviation of the average mean square fit statistic (Wright & Stone, 2004). Inspecting the

outfit mean-squares provides evidence about the fit of the data to the model. The infit mean-

squares were used to determine the fit of the item within the construct.

Misfit is indicated by an item’s infit mean square fit statistic above 1.23, while overfit is

indicated by an infit mean square fit statistic less than 0.79. An infit mean square above 1.23

illustrates that the item does not fit according to the response patterns of other items. An infit

mean square less than 0.79 indicates the item response fits the model and little distinction can be

made about the respondents based on the item. Ten items in the overall analysis have fit statistics

that lie outside the suggested range of one standard deviation above the average infit or outfit

statistic, which was 1.23 and 1.19 respectively (see Table 5). Nine out of ten of these items were

within the methodology section of the survey. Nine items overfit the model, meaning the item fit

statistics were over one standard deviation below the infit or outfit statistic which was 0.79 and

0.77 respectively (see Table 6).



Rating scale category function

For the survey instrument, the responses correspond to the scale where 1 = Strongly

Disagree, 2 = Disagree, 3 = Agree and 4 = Strongly Agree. The number of observations and

distribution of observations across categories were examined to describe the functioning of the

rating scale categories. The observed count indicates the number of times the category was

selected (see Table 7).




22

The observed count in the category measure does increase with the rating scale as

expected. It appears from the frequencies reported that respondents are utilizing the full range of

the four-point scale. The majority of responses in the overall model lie in the agreement

categories, which indicates most survey items were likely to be endorsed by the average

participant. Advancing average measures with each category and step calibrations ensure the

rating scale measure is stable and accurate. Probability curves were used to visually inspect the

rating scale category function. The probability curves representing the function of rating scale

categories demonstrated respondents utilizing the full scale. All rating categories reached a peak

within 3 logits, which indicates respondents used all four response categories.

Empirical characteristics of responses

The item map produced from the Rasch analysis provides an empirical hierarchy for

survey items based on responses, which represents characteristics of good research that are least

to most likely to receive endorsement. The item map revealed the average person measure was

above the average item measure, which indicates the majority of the survey items were more

likely to be endorsed by participants (see Figure 1).

[Insert Figure 1 Here]

A large gap exists between survey item M2 and survey items M12, M13, M14, and M15.

The survey items 12 and 15 from the methodology section were the only two items over one

standard deviation above the average person measure. Overall, participants do not agree that

these items describe characteristics of high quality research (see Table 8).


Summary



23

The overall fit of the data to the Rasch model was acceptable. The methodology

component of the survey resulted in the most misfitting items as well as the most difficult items

to endorse. Participants were distributed across all primary research methodologies, which

accounts for some discrepancies across the methodology survey items. The results provide a

foundation to guide revisions to the instrument prior to the full scale study.

Discussion

Prior to interpreting participant responses to statements concerning what constitutes good

educational research, the survey instrument itself was examined. In review of the quality of the

measure, the survey was evaluated for fit of data to the Rasch model, orientation of the latent

variable to ensure directional polarity, and functionality of the rating scale response categories.

Based on person and item reliability and separation estimates (.86 and .98; 2.48 and 7.64,

respectively), the survey instrument appears to be a reasonably reliable measure of educational

researchers' conception of what constitutes high-quality research. Point-biserial correlations

demonstrated polarity of the items in the positive direction, ensuring unidimensionality of the

survey and indicating good construct validity.

Concerning the fit of data to the one-parameter IRT Rasch model, it appears that ten

items were slightly misfitting, as indicated by infit or outfit mean square fit statistics one

standard deviation above the suggested range. Misfitting items draw survey-developers attention

to the function of these items and patterns of participant responses that do not match what is

expected. Specific to this survey, all misfitting items were methodological statements. Similarly,

nine items overfit the model, as indicated by fit statistics one standard deviation below the

suggested range. Overfitting items may provide researchers with minimal information and little

distinction between groups of participants based on such items. Items that appeared to have



24

overfit included all domains (ethical, methodological and theoretical). In each instance where

item data does not fit the model as intended, researchers must take a closer look at the item and

make revisions as appropriate. In the case of misfitting methodological items, however,

discrepancies in responses may be accounted for by the differences in participants' preferred

methodological approach. Another explanation may be that methodological issues are not well

articulated by the items themselves. Therefore, potentially misfitting items may need revision

rather than be taken off the survey in their entirety.

With regard to the function of the rating scale response categories, multiple analytical

approaches were taken to evaluate participant utilization of responses for items. Observed count,

frequency statistics, and probability curves demonstrate that participants utilized the full range of

the four-point scale. Although participants mostly endorsed agreement to statements about high-

quality research characteristics, each category was endorsed, ensuring that the rating scale

measure is both stable and accurate.

Person and item maps indicated which characteristics were most likely to be endorsed by

participants, thus indicating which characteristics of good educational research were least or

most salient to respondents. Overall, participants disagreed with items M12 ("High-quality

research requires random sampling") and M15 ("High-quality research can be determined solely

by examining the research methodology"). Similarly, participants showed little agreement with

items M13 ("High-quality research requires quantifiable measures of results") and M14 ("High-

quality research consists of experimental studies that yield prescriptions for actions"). Closely

related to explanations for item misfit, it is reasonable to argue that these questions were too

methodologically-based to provoke agreement by both quantitative and qualitative researchers.

On the other hand, participants generally agreed to all other items in the survey. Characteristics



25

with the most participant agreement included: E3 (“High-quality research should protect the

safety and welfare of participants.”), E1 (“High-quality research abides by ethical standards.”),

and M6 (“High-quality research should be methodologically sound.”).

Ethical Considerations

Faculty and student participant responses to ethics items revealed a number of opinions

reasonably aligned with the available literature regarding ethics and research. Respondents found

it less difficult to endorse compliance with unspecified ethical standards (E1), and similarly,

adherence to guidelines recognized by their professional organizations (E10). These results could

reflect an appreciation of the ethical guidelines, currently in existence, imparted by professional

associations such as the AERA and the APA. Another explanation may be that researchers in

educational disciplines generally agree with generic ethical guidelines regardless of preferred

methodological approach, but may differ according to methodology in reference to procedure-

specific characteristics of good research, as seen in responses to other, more methods-based

items.

Likewise, further results suggested that responses to items regarding recognition of

contributors to research (E6), safety and welfare of participants (E3), adherence to institutional

policies (E5), and minimal use of techniques yielding negative social consequences (E4), were

more difficult to endorse yet still more likely to evoke positive responses. These responses

reflect the content of ethical guidelines offered by professional organizations, as well as those

presented in social research texts, relating to informed consent, credit, and the use of Institutional

Review Boards.

Respondents also found it less difficult to endorse item E7, which addresses execution of

research by investigators who have completed ethics training. These results could be interpreted



26

as representative of knowledge of the branch of educational research literature devoted to calls

for increased ethics education. Additional items or perhaps a better description of ethics training

and education could enhance interpretation of these results.

Items E8 and E7 related to dissemination of research findings to professionals in the

field, as well as to the public were found to be more difficult for participants to endorse.

Recalling Hostetler’s (2005) argument regarding how research should serve the public’s welfare,

these findings are worthy of note. This brings into question how participants interpreted the

questions. Contrary to professional ethical guidelines regarding dissemination of information,

respondents found these items more difficult to endorse. Do faculty and students believe that

research, in general, might not always necessitate dissemination or are there specific instances in

which research should be held from other professionals or the public? Are researchers concerned

with the implications that certain results may have on practice? Refining the instrument with

additional items could lead to a greater understanding of the motives behind these responses.

In his discussion of the topic of what is “good” research, Hostetler (2005) stated, “Good

education research is a matter not only of sound procedures but also of beneficial aims and

results; our ultimate aim as researchers and educators is to serve people’s well being” (p.16).

One could suggest the findings of this study reveal the importance of ethical considerations in

research practice. When looking at the items participants found less difficult to endorse, the

majority of the items were found in the section of ethical considerations. As stated earlier, item

E3 (safety and welfare of participants) was the item that participants found easy to endorse.

Hostetler would concur with the high endorsability of this item due to researchers “respecting the

dignity and human of persons” (p.19). Because participants found items regarding ethical

practices as less difficult to endorse, a trend could be observed about the importance of ethical



27

practices being taught in the classroom and in the field of research.

In general, responses positively reflected the current ethical guidelines offered by

professional organizations. Modification of the survey to include items related to ethics

education would be more representative of the current body of literature, and perhaps, assist in

filling the gap between calls for more ethics education and the relationship between that

education and how it might better inform practice. Additional items regarding research

dissemination could yield a stronger understanding of how respondents interpreted these

questions, and why these items appear more difficult to endorse than others.

Methodological Considerations

An educational researcher’s choice of procedures and methodology is influenced by his

or her underlying philosophical assumptions about reality, knowledge and values. Quantitative

and qualitative approaches to research are often differentiated by the type of data collected

(numeric or descriptive), the mode of analysis (statistical or interpretive), the logic employed

(deductive or inductive) and underlying paradigms (positivism or constructivism) (Tashakkori &

Teddlie, 1998; Merriam, 1998; Creswell & Plano-Clark, 2007). Thus, the variation in participant

agreement among survey items M12 ("High-quality research requires random sampling") and

M13 ("High-quality research requires quantifiable measures of results") is reasonable as

qualitative and quantitative researchers preferred sampling procedures and types of data

collected are distinct. The research question(s) are also an influential factor contributing to a

researcher’s choice of procedures and methodology employed. A carefully constructed alignment

of research methods to research question(s) is essential if one wishes to elucidate useful and

credible findings to the research question(s) (Johnson & Onwuegbuzie, 2004; Sikes, 2004).

Results revealed that participants generally agreed with survey items M6 (“High-quality research



28

should be methodological sound”) and M11 (“High-quality research requires research methods

and techniques based on the nature of the research questions”). These results indicate participant

responses align well with the existing research literature on the importance of choosing sound

methods and procedures that will best address the research question(s).

Theoretical Considerations

When viewing the findings from the section on theoretical considerations, there are

multiple connections between the findings and the literature. With regard to item fit, the analysis

revealed six items in this section that were overfitting ( T1, T7, T10, T11, & T12). The

overfitting of these items suggests that they can be used to measure the construct, although there

is not much of a distinction between the items. Items T10 (“High-quality research should be

useful to other professionals), T11 (“High-quality research connects the work to its impact on

human-well being”, and T12 (“High-quality research merges reason and value) appear to attest to

the expansion of knowledge within and across disciplines. As presented in the literature review,

theory can be viewed as an action guide in research applied to predict certain outcomes.

Participants found items T2 (“High-quality research should provide the rationale for the

conceptual orientation of the study”) and T3 (“High-quality research should provide the rationale

for the theoretical orientation of the study”) as less difficult to endorse which supports the notion

of implementing theory as the guiding factor of high-quality research. Also, participants

identified items T4 (“Educational researchers should adhere to the standards of their own

theoretical perspectives to achieve high-quality research”), T5 (“Reliability, validity, and

trustworthiness are the most important considerations in high-quality research”), and T6 (“High-

quality research provides objective answers to research questions”) as items more difficult to

endorse. The higher level of difficulty could be the result of items having a strong correlation to



29

professional ethics (T4) or methodological practices (T5 and T6). Overall, results indicate that

the participants appear to be cognizant of the importance of theory in high-quality research.

Conclusion

While it seemed reasonable to seek consensus among the community of education

researchers of what constitutes good research, evidence from this study supports that variability

exists across the general themes of standards in conducting research in education. This study

aimed to operationalize good education research as it pertains to three major domains (Ethics,

Methods, and Theory) by using AERA and APA as guiding authorities. The primary expectation

was to reveal which characteristics are most frequently endorsed by members of various

disciplines within educational research through examining relative frequency tables as well as

the empirical hierarchy of survey items illustrating item endorsability. The secondary purpose

was to investigate and illustrate the fit of the data to a Rasch measurement model to support the

quality of the survey measure employed and to guide revisions prior to the full scale study.

While a consensus was not determined as to what constitutes good research, attention

should be given to the characteristics that did emerge as highly endorsed items as well as items

that participants were less likely to endorse. All the items within the ethics and theory domains

were empirically found to be relatively easy to endorse for the majority of participants; however,

more variation existed in terms of item endorsability within the methods domain. Survey items

M12, M13, M14, and M15 were the most difficult items for participants to endorse. Of these,

items M12 and M13 were also highlighted as problematic items in terms of fit to the model and

will be further reviewed.

As revisions to the survey instrument are made, further study is needed to determine

variations in what constitutes good research in colleges of education across the nation. This study



30

stimulates dialogue how each characteristic functions as a driving force for education research,

as well as whether each should be a foundation for the research that educational researchers

conduct. Results will further challenge researchers to be mindful of characteristics of their own

research, which will ultimately drive better research in education across disciplines.



31

References

American Educational Research Association (2000). Ethical Standards of the American

Educational Research Association: Retrieved August 15, 2007, from

http://www.aera.net/uploadedFiles/About_AERA/Ethical_Standards/EthicalStandards.

pdf

American Psychological Association (2002). Ethical Principles of Psychologists and Code

of Conduct. Retrieved August 15, 2007, from http://www.apa.org/ethics/code2002.pdf

Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43,

561-574.

Babbie, E. (2008). The basics of social research. Belmont, CA: Thomson Wadsworth.

Bond, T., & Fox, C. (2001). Applying the Rasch model: Fundamental measurement in the human

sciences. Mahwah, NJ: Lawrence Erlbaum Associates.

Brandon, D. (2000). What is research? Retrieved August 15, 2007, from

http://www.scitech.technion.ac.il/faq/research.html

Bridges, D. (2001).The ethics of outsider research. Journal of Philosophy of Education. 35, 371-

386.

Bruhn, J., Zajac, G., Al-Kazemi, A., & Prescott, Jr., L. (2002). Moral positions and academic

conduct: Parameters of tolerance for ethics failure. The Journal of Higher Education, 73,

461-493.

Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches (2nd Ed.). Thousand Oaks, CA: SAGE Publications, Inc.

Creswell, J., & Plano-Clark, V. L. (2007). Designing and conducting mixed methods research.

Thousand Oaks, CA: SAGE Publications, Inc.



32

Creswell, J. W., Plano-Clark, V. L., Gutmann, M., & Hanson, W. E. (2002). Advanced

mixed methods research designs. In A. Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 209-240).

Thousand Oaks, CA: Sage.

Ebbs, C. (1996). Qualitative research inquiry: Issues of power and ethics. Education. 117,

217-223.

Elmes, D. G., Kantowitz, B. H., & Roediger III, H. L. (1996). Research Methods in Psychology,

Fifth Edition. New York: West Publishing Company.

Hostetler, K. (2005). What is “good” education research? Educational Researcher, 34(6), 16-21.

Howe, K., & Eisenhart, M. (1990). Standards for qualitative (and quantitative) research: A

prolegomenon. Educational Researcher, 19(4), 2-9.

Hughes, W. P. (1999). Constructions research: A field of application. Australian Institute of

Building Papers, 9, 51-58.

Iutcovich, J., Kennedy, J., & Levine, F. (2003). Establishing an ethical climate in support

of research integrity: Efforts and activities of the American Sociological

Association. Science and Engineering Ethics, 9, 201-205.

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm

whose time has come. Educational Research, 33(7), 14-26.

Kansanen, P. (2003). Pedagogical ethics in educational research. Educational Research and

Evaluation. 9, 9-23.

Kerse, N. & Elley, R. (2003). What constitutes good research in general practice? The New

Zealand Family Physician, 30 (6), 385-387.



33

Linacre, J. M. (2004). Optimizing rating scale category effectiveness. In E. V. Smith, Jr. and R.

M. Smith Introduction to Rasch Measurement (pp. 258-278) Maple Grove, MN: JAM

Press.

Linacre, J. M. (2005). WINSTEPS Rasch measurement computer program. Chicago:

Winsteps.com.

Merriam, S.B. (1998). Qualitative research and case study applications in research. San

Francisco, CA: Jossey-Bass.

Muskavitch, K. (2005).Cases and goals for ethics education. Science and Engineering

Ethics, 11, 431-434.

Nardi, P. (2003). Doing survey research: A guide to quantitative methods. Boston, MA:

Pearson Education, Inc.

Simmerling, M., Schwegler, B., Sieber, J., & Lindgren, J. (2007). Introducing a new

paradigm for ethical research in the social, behavioral, and biomedical sciences:

Part I. Northwestern University Law Review, 101, 837-859.

Sikes, P. (2004). Methodology, procedures and ethical concerns. In C. Opie (Ed.), Doing

education research (pp. 15-33). Thousand Oaks, CA: SAGE Publications, Inc.

SurveyMonkey [Survey software program] (2007). Copyright 1999-2007 SurveyMonkey.com.

Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and

quantitative approaches. Thousand Oaks, CA: SAGE Publications, Inc.

Whitley Jr, B. E. (1996). Principles of Research in Behavioral Science. Mountain View, CA:

Mayfield Publishing Company.

Wright, B. D., & Masters, G. N. (1982). Rating scale analysis: Rasch measurement. Chicago, IL:

MESA Press.



34

Wright, B. D., & Stone, M. H. (2004). Making Measures. Chicago, IL: The Phaneron Press.



35

Table 1 Personal Demographic Characteristics of Faculty Respondents (n = 50) Characteristic n % Faculty Rank

Instructor 3 5.9 Lecturer 0 0 Assistant Professor 15 29.4 Associate Professor 12 23.5 Professor 13 25.5 Visiting Professor 2 3.9 Adjunct Professor 1 2.0 Part-time Professor/Instructor 2 3.9 Other 3 5.9

Years as a faculty member (at any higher education institution)

Less than 1 year 6 12.0 1-3 years 12 24.0 4-6 5 10.0 7-9 3 6.0 10-15 7 14.0 16-20 4 8.0 Greater than 20 years 13 26.0

Tenure Status

Tenured 27 52.9 On tenure track, but not tenured 13 25.5 Not on tenure track 11 21.6

Age 71 or older 2 4.3 61-70 6 12.8 51-60 12 25.5 41-50 7 14.9 31-40 15 31.9 Less than 30 5 10.6



36

Table 2 Personal Demographic Characteristics of Graduate Student Respondents (n = 76) Characteristic n % Status Graduate Assistant 3 4.1

Research Assistant 19 25.7 Teaching Assistant 14 18.9 Doctoral Student 39 52.7 Master’s Student 8 10.8 Specialist Degree Student 8 10.8 Other 12 16.2

Years as a graduate student (at any institution)

Less than a year 8 11.0 Between 1-2 years 14 19.2 Between 3-4 years 21 28.8 Between 5-6 years 18 24.7 7 or more years 12 16.4

Age Under 20 0 0

20-24 12 16.0 25-29 23 30.7 30-34 10 13.3 35-39 10 13.3 40-44 5 6.7 45-49 5 6.7 50-54 6 8.0 55 or older 4 5.3



37

Table 3 Primary Academic Interest for Faculty and Graduate Student Respondents (n = 121) Educational area n % Administration, Organization & Leadership 6 5.0 Curriculum Studies 3 2.5 Learning & Instruction 21 17.4 Measurement & Research 8 6.6 Counseling & Human Development 14 11.6 History & Historiography 2 1.7 Social Context of Education 10 8.3 School Evaluation & Program Development 5 4.1 Education in the Professions 3 2.5 Postsecondary Education 7 5.8 Teaching & Teacher Education 17 14.1 Educational Policy & Politics 6 5.0 Other 19 15.7



38

Table 4 Interests of Faculty and Graduate Student Respondents (n = 76) Item n % Primary interest in teaching or research

Very heavily in research 11 9.2 In both, but leaning toward research 41 34.2 In both, but leaning toward teaching 57 47.5 Very heavily in teaching 11 9.2

Hours spent on research each week

None 13 10.5 1-9 57 46.0 10-19 30 24.2 20-29 14 11.3 30-39 3 2.4 40 or more 7 5.7

Primary research methodology

Entirely qualitative 4 3.4 Mostly qualitative 25 21.2 Mixed methods (50/50) 41 34.8 Mostly quantitative 31 26.3 Entirely quantitative 17 14.4

Perception of Educational Research Quality

Very low quality 1 0.8 Low quality 14 11.7 Average quality 69 57.5 High quality 35 29.2 Very high quality 1 0.8



39

Table 5

Item fit statistics poorly fitting the model Item Number Count Measure Infit Outfit

M10 136 0.86 1.38 1.40

M12 136 2.91 1.40 1.40

M3 137 -0.26 1.36 1.30

M6 137 -1.77 1.31 1.07

M2 135 1.39 1.18 1.31

M17 133 0.74 1.27 1.30

M4 137 -1.33 1.25 1.07

M13 136 2.43 1.25 1.25

M1 136 -0.54 1.25 1.18

E4 127 -0.16 1.20 1.23



40

Table 6

Item fit statistics overfitting the model Item Number Count Measure Infit Outfit

T1 126 -0.47 0.78 0.83

T11 127 0.32 0.80 0.78

M16 136 -0.35 0.78 0.76

T7 125 0.39 0.73 0.73

T10 127 -0.42 0.73 0.69

E9 127 0.33 0.71 0.72

E10 129 -1.01 0.67 0.69

T9 127 -0.44 0.68 0.66

T12 123 0.35 0.63 0.65

Table 7

Summary of Rating Scale Diagnostics Category Observed INFIT OUTFIT

Count (%) Mean Square Mean Square Strongly Disagree 175 (3%) 1.58 1.77 Disagree 715 (13%) 0.87 0.82 Agree 2167 (41%) 0.94 0.87 Strongly Agree 2042 (38%) 0.99 0.99

Note. Category, observed count, and percentage indicate the numbers of respondents who chose a particular response category, summed for each category across all 39 items.



41

Figure 1. Map of survey items by difficulty to endorse

<more>|<difficult to endorse> | | 4 . + # | # T| | # | 3 .## + M12 M15 .## | ### S|T #### | M13 M14 .## | 2 ###### + ###### M| .####### | .###### | M2 ######## |S T8 1 .####### + T6 .## S| M10 M17 M5 ### | T4 T5 ## | E9 T11 T12 T7 . | E8 0 T+M . | E4 E7 M3 T3 | M16 T1 T10 T9 | E5 M1 T2 | E6 M11 -1 + E10 |S E2 M7 M8 | M4 M9 | | M6 -2 + E1 | E3 | |T | -3 + <less>|<difficult to endorse> EACH '#' IS 2.



42

Table 8

Counts and percents of responses for highlighted survey items No. 1 2 3 4 Survey Item Description

M12 37 (27%) 70 (51%) 22 (16%) 7 (5%) High-quality research requires random sampling.

M13 27 (20%) 60 (44%) 42 (31%) 7 (5%) High-quality research requires quantifiable measures of results.

M14 24 (18%) 63 (47%) 41 (30%) 7 (5%) High-quality research consists of experimental studies that yield prescriptions for actions.

M15 29 (21%) 92 (68%) 13 (10%) 2 (1%) High-quality research can be determined solely by examining the research methodology.

Documents

Perceptions of Education Research 1 Running head: GOOD ...kdbrad2/MWERA_GoodResearch.pdfPerceptions of Education Research *** Working Draft *** Do not cite without permission of contact

Perceptions of Education Research 1 Running head: GOOD ...kdbrad2/MWERA_GoodResearch.pdfPerceptions of Education Research * Working Draft * Do not cite without permission of contact