Report on Class Presentation synopsis

8/6/2019 Report on Class Presentation synopsis

http://slidepdf.com/reader/full/report-on-class-presentation-synopsis 1/9

REPORT ON CLASS PRESENTATION

Topic of presentation:

Validity in business research methods

Report Submitted to:

S ir Mazhar Manzoor

Report prepared by:

Mohammad Irfan (MBA3 SEC.µ A µ R .NO#51)

Subject of presentation:

Business Research Methods

FEDERAL URDU UNIVERSITY KARACHI GULSHAN CAMPUS



Validity is the extent to which a test measures what it claims to measure. It is vital for a test to be valid in order for the results to be accurately applied and interpreted.

Validity isn¶t determined by a single statistic, but by a body of research thatdemonstrates the relationship between the test and the behavior it is intended tomeasure. There are three types of validity:

Content validity:

When a test has content validity, the items on the test represent the entire range of possible items the test should cover. Individual test questions may be drawn from alarge pool of items that cover a broad range of topics.

In some instances where a test measures a trait that is difficult to define, an expert judge may rate each item¶s relevance. Because each judge is basing their rating onopinion, two independent judges rate the test separately. Items that are rated asstrongly relevant by both judges will be included in the final test.

Criterion-related Validity:

A test is said to have criterion-related validity when the test has demonstrated itseffectiveness in predicting criterion or indicators of a construct. There are twodifferent types of criterion validity:

y Concurrent Validity occurs when the criterion measures are obtained at thesame time as the test scores. This indicates the extent to which the test scoresaccurately estimate an individual¶s current state with regards to the criterion. For

example, on a test that measures levels of depression, the test would be said tohave concurrent validity if it measured the current levels of depressionexperienced by the test taker.

y Predictive Validity occurs when the criterion measures are obtained at a timeafter the test. Examples of test with predictive validity are career or aptitude tests,which are helpful in determining who is likely to succeed or fail in certain subjectsor occupations.

Construct Validity:

A test has construct validity if it demonstrates an association between the test scoresand the prediction of a theoretical trait. Intelligence tests are one example of measurement instruments that should have construct validity.

Test validity for a pre-employment is a study undertaken and directed by the testpublisher in accordance with certain professional st andards to measure what it is



supposed to measure. There are two main methods to demonstrate a test¶s validity,one is criterion validity and the other is content validity.

Criterion validity assesses whether a test reflects a certain set of abilities. For instance, when an employer hires new employees based on normal hiringprocedures like interviews, education, and experience. This method demonstratesthat people who do well on a test will do well on a job, and people with low score ontest will do poorly on a job.

Content validity represents job function testing. The procedure here is to identifynecessary tasks to perform a job like typing, design, or physical ability. In order todemonstrate the content validity of a selection procedure, the behaviorsdemonstrated in the selection should be a representative sample of the behaviors of the job.

Another method that is used rarely because it is not very sophisticated is facevalidity. Face validity is the property of a test intended to measure something. It is

based only on the appearance of the measure and what it is supposed to measure,but not what the test actually measures. Face validity is often contrasted with contentvalidity because it is not validity in the technical sense.

Face validity is a simple form of validity in which researchers determine if the testseems to measure what is intended to measure. Essentially, researchers are simplytaking the validity of the test at face value by looking at whether a test appears to

measure the target variable. On a measure of happiness, for example, the test wouldbe said to have face validity if it appeared to actually measure levels of happiness.

Obviously, face validity only means that the test looks like it works. It does not mean

that the test has been proven to work. However, if the measure seems to be valid atthis point, researchers may investigate further in order to determine whether the testis valid and should be used in the future.

VALIDITY

In general, VALIDITY is an indication of how sound your research is. Morespecifically, validity applies to both the design and the methods of your research.Validity in data collection means that your findings truly represent the phenomenon

you are claiming to measure. Valid claims are solid claims.

Validity is one of the main concerns with research."Any research can be affected by different kinds of factors which, while extraneous to the concerns of theresearch, can invalidate the findings" (Seliger &Shohamy 1989, 95). Controlling all possible factorsthat threaten the research's validity is a primaryresponsibility of every good researcher.



INTERNAL VALIDITY is affected by flaws within the study itself such as notcontrolling some of the major variables (a design problem), or problems with theresearch instrument (a data collection problem).

"Findings can be said to be internally invalid because they may have been affectedby factors other than those thought to have caused them, or because the

interpretation of the data by the researcher is not clearly supportable" (Seliger &Shohamy 1989, 95).

Here are some factors which affect internal validity:

y Subject variabilityy Size of subject populationy Time given for the data collection or experimental treatmenty Historyy Attritiony Maturationy Instrument/task sensitivity

EXTERNAL VALIDITY is the extent to which you can generalize your findings to alarger group or other contexts. If your research lacks external validity, the find ingscannot be applied to contexts other than the one in which you carried out your research. For example, if the subjects are all males from one ethnic group, your findings might not apply to females or other ethnic groups. Or, if you conducted your research in a highly controlled laboratory environment, your findings may notfaithfully represent what might happen in the real world.

"Findings can be said to be externally invalid because [they] cannot be extended or applied to contexts outside those in whic h the research took place" (Seliger &

Shohamy 1989, 95).

Here are seven important factors affect external validity:

y Population characteristics (subjects)y Interaction of subject selection and researchy Descriptive explicitness of the independent variabley The effect of the research environment



y Researcher or experimenter effectsy Data collection methodologyy The effect of time

Validity: the best available approximation to the truth of a given proposition,

inference, or conclusion

The first thing we have to ask is: "validity of what ?" When we think about validity in

research, most of us think about research components. We might say that a

measure is a valid one, or that a valid sample was drawn, or that the design had

strong validity. But all of those statements are technically incorrect. Measures,

samples and designs don't 'have' validity -- only propositions can be said to be valid.

Technically, we should say that a measure leads to valid conclusions or that a

sample enables valid inferences, and so on. It is a prop osition, inference or

conclusion that can 'have' validity.

We make lots of different inferences or conclusions while conducting research. Many

of these are related to the process of doing research and are not the major

hypotheses of the study. Nevertheless, like the bricks that go into building a wall,

RESEARCH VALIDITY

Validity is the extent to which a question or scale is measuring the concept,attribute or property it says it is.

For example, if one is measuring an attribute of a product such as "BrandRecognition", how do we know that the question (or questions) used to measureBrand Recognition are valid? Usually, this is a result of what we know of the"perceived meaning" of the question from previous times it has been used.Validity can also be optimized by careful pretesting of alternative questions

designed to measure the same concept.In the academic sphere, validity can take many forms. Construct validity for example refers to the ability of a measure to relate meaningfully to other similar measures used before. In commercial market research, many companies rely on"face validity" - the extent to which the respondent "knows" what is beingmeasured and it seems sensible to them. Validity can be differentiated fromReliability, another property of "good" research .

In practice, validity can also refer to the success of the project in retrieving "valid"results. There are many sources of error that can reduce the validity of a projectincluding poor sample selection and resultant bias, simple coding errors,misunderstanding of management and research questions by the researchers

and misunderstanding of the investigative questions by the respondents. Other errors include asking "leading questions", unconscious non -verbal prompts on"good answers", vindictive respondents, or inappropriate methodologies used toanalyze the raw data.

Competent market research agencies emphasize validity of results in their research conduct, even over presentation and colorful reports, which cansometimes obfuscate the actual validity of a market research survey.



these intermediate process and methodological propositions provide the foundation

for the substantive conclusions that we wish to address. For instance, virtually all

social research involves measurement or observation. And, whenever we measure

or observe we are concerned with whether we are measuring what we intend to

measure or with how our observations are influenced by the circumstances in whichthey are made. We reach conclusions about the quality of our measures --

conclusions that will play an important role in addressing the broader substantive

issues of our study. When we talk about the validity of research, we are often

referring to these to the many conclusions we reach about the quality of different

parts of our research methodology.

We subdivide validity into four types. Each type addresses a specific methodological

question. In order to understand the types of validity, you have to know something

about how we investigate a research question. Because all four validity types are

really only operative when studying causal questions, we will use a causal study to

set the context.

The figure shows that there are really two realms that are involved in research. The

first, on the top, is the land of theo ry. It is what goes on inside our heads as

researchers. It is we keep our theories about how the world operates. The second,

on the bottom, is the land of observations. It is the real world into which we translate

our ideas -- our programs, treatments, measures and observations. When we



conduct research, we are continually flitting back and forth between these two

realms, between what we think about the world and what is going on in it. When we

are investigating a cause-effect relationship, we have a theory (implicit or otherwise)

of what the cause is (the cause construct ). For instance, if we are testing a new

educational program, we have an idea of what it would look like ideally. Similarly, onthe effect side, we have an idea of what we are ideally trying to affect and measure

(the effect construct ). But each of these, the cause and the effect, has to be

translated into real things, into a program or treatment and a measure or

observational method. We use the term operationalization to describe the act of

translating a construct into its manifestation. In effect, we take our idea and describe

it as a series of operations or procedures. Now, instead of it only being an idea in our

minds, it becomes a public entity that anyone can look at and examine for

themselves. It is one thing, for instance, for you to say that you would like to

measure self-esteem (a construct). But when you show a ten-item paper-and-pencil

self-esteem measure that you developed for that purpose, others can look at it and

understand more clearly what you intend by the term self -esteem.

Now, back to explaining the four validity types. They build on one another, with two

of them (conclusion and internal) referring to the land of observation on the bottom of

the figure, one of them ( construct) emphasizing the linkages between the bottom and

the top, and the last (external) being primarily concerned about the range of our

theory on the top. Imagine that we wish to examine whether use of a World WideWeb (WWW) Virtual Classroom improves student understanding of course material.

Assume that we took these two constructs, the cause construct (the WWW site) and

the effect (understanding), and operationalIized them -- turned them into realities by

constructing the WWW site and a measure of knowledge of the course material.

Here are the four validity types and the question each addresses:

Conclusion Validity: In this study, is there a relationship between the two variables?

In the context of the example we're considering, the question might b e worded: inthis study, is there a relationship between the WWW site and knowledge of course

material? There are several conclusions or inferences we might draw to answer such

a question. We could, for example, conclude that there is a relationship. We might

conclude that there is a positive relationship. We might infer that there is no

relationship. We can assess the conclusion validity of each of these conclusions or

inferences.



Internal Validity: Assuming t hat t here is a relationship in t his study, is the relationship

a causal one?

Just because we find that use of the WWW site and knowledge are correlated, we

can't necessarily assume that WWW site usec auses the knowledge. Both could, for

example, be caused by the same factor. For instance, it may be tha t wealthier

students who have greater resources would be more likely to use have access to a

WWW site and would excel on objective tests. When we want to make a claim that

our program or treatment caused the outcomes in our study, we can consider the

internal validity of our causal claim.

Construct Validity: Assuming t hat t here is a c ausal relationship in t his study , can we

claim that the program reflected well our construct of the program and that our

measure reflected well our idea of the construct of the measure?

In simpler terms, did we implement the program we intended to implement and did

we measure the outcome we wanted to measure? In yet other terms, did we

operationalize well the ideas of the cause and the effect? When our research is over,

we would like to be able to conclude that we did a credible job of operational zing our

constructs -- we can assess the construct validity of this conclusion.

External Validity: Assuming t hat t here is a c ausal relationship in t his study bet w een

t he c onstruc ts of t he c ause and t he effec t ; can we generalize this effect to other persons, places or times?

We are likely to make some claims that our research findings have implications for

other groups and individuals in other settings and at other times. When we do, we

can examine

the external

validity of

these claims.

Notice how

the question

that each

validity type

addresses



presupposes an affirmative answer to the previous one. This is what we mean when

we say that the validity types build on one another. The figure shows the idea of

cumulativeness as a staircase, along with the key question for each validity type.

For any inference or conclusion, there are always possible threats to validity --reasons the conclusion or inference might be wrong. Ideally, one tries to reduce the

plausibility of the most likely threats to validity, thereby leaving as most plausible the

conclusion reached in the study. For instance, imagine a study examining whether

there is a relationship between the amount of training in a specific tech nology and

subsequent rates of use of that technology. Because the interest is in a relationship,

it is considered an issue of conclusion validity. Assume that the study is completed

and no significant correlation between amount of training and adoption ra tes is

found. On this basis it is c onc luded that there is no relationship between the two.

How could this conclusion be wrong -- that is, what are the "threats to validity"? For

one, it's possible that there isn't sufficient statistical power to detect a relationship

even if it exists. Perhaps the sample size is too small or the measure of amount of

training is unreliable. Or maybe assumptions of the co relational test are violated

given the variables used. Perhaps there were random irrelevancies in the stu dy

setting or random heterogeneity in the respondents that increased the variability in

the data and made it harder to see the relationship of interest. The inference that

there is no relationship will be stronger -- have greater conclusion validity -- if one

can show that these alternative explanations are not credible. The distributions mightbe examined to see if they conform with assumptions of the statistical test, or

analyses conducted to determine whether there is sufficient statistical power.

The theory of validity, and the many lists of specific threats, provides a useful

scheme for assessing the quality of research conclusions. The theory is general in

scope and applicability, well -articulated in its philosophical suppositions, and virtually

impossible to explain adequately in a few minutes. As a framework for judging the

quality of evaluations it is indispensable and well worth understanding.

Links

http://coles.kennesaw.edu/drbob/dr.bob/Introduction%20to%20Validity.pdf

http://www.asiamarketresearch.com/glossary/validity.htm

http://psychology.about.com/od/researchmethods/f/validity.htm

Documents

Report on Class Presentation synopsis