Comprehensive Exam ReviewClick the LEFT mouse key ONCE to continue

Research andProgram EvaluationPart 1 Click the LEFT mouse key ONCE to continue

Types of Research

Theoretically, basic research is conducted solely for the purpose of developing and/or refining theory.Theoretically, applied research is conducted solely for the purpose of evaluating the application of theory for the solution of problems.This distinction is not, however, a particularly helpful way to consider research methodologies, because basic and applied are actually extremes on a continuum of application and generaliza-bility to other situations.

The two most commonly discussed types of research are qualitative and quantitative.Technically, qualitative and quantitative are different approaches to inquiry rather than distinctly different research methodology categories.However, the qualitative / quantitative differ-entiation facilitates explanation of different research methodologies.

Qualitative Research

Qualitative research involves collection of extensive data on many variables over an extended period of time in a naturalistic setting.Qualitative research methodologies are grounded in the belief that behavior is significantly influenced by the environment in which it occurs.Qualitative researchers seek to find holistic, in-depth understanding of the phenomenon observed.

Qualitative research is inductive, and its primary goal is to promote greater understanding by explaining how and why people behave the way they do.Measurement; data collection, analysis, and interpretation techniques; and research design and method in qualitative research are flexible and evolve as the research process proceeds.

Participant or nonparticipant observation are the most frequently used research strategies in qualitative research, particularly in the social sciences.Qualitative research may involve interactive strategies, such as participant observation, or noninteractive strategies, such as review of documents.

The primary distinguishing characteristic of qualitative research is that the actual methodology used cannot be determined a priori; the methodology emerges quite literally during the process and often involves both interactive and noninteractive strategies.Qualitative research usually involves pur-poseful sampling, as opposed to probabilistic sampling, to ensure that the right person(s) and/or situation(s) are examined.

Generalizability of the results is not a major concern in qualitative research (in accord with nonprobabilistic sampling), and usually generalizability is very restricted.A qualitative research case study design involves focus on one phenomenon, such as a person, a concept, a process, a group, or a program.

Qualitative case study designs are often used to:serve as a prelude to quantitative research.describe and analyze a situation, event, orprocess,develop a concept or model,evaluate a program,investigate social and cultural beliefs, or

Ethnographic research is sometimes con-sidered synonymous with qualitative research and sometimes as a subtype of qualitative research.In the latter perspective, ethnographic research is usually interpreted to mean the study of a culture.In this context, a culture is any group of people who regularly associate with one another and who develop characteristic ways of behaving and thinking.

Other key concepts in qualitative research include:Participant observation, in which the re-searcher literally becomes a participant in a situation and makes research observations while participating.Observer effect, in which the presence of the participant observer alters the nature of the situation.

Observer bias, in which the participant observer makes inaccurate and/or invalid interpretations of the phenomenon being observed.Fieldnotes, which are the actual data resulting from the participant observation process.Grounded theory, which is theory (or com-ponents of theory) developed from the data collected in real world settings.

Peer debriefer, which is a colleague who works with the primary researcher to generate meaning from the data collected.Audibility, which is the process of maintaining a record of data management techniques that document the decision trail used.Key informant, which is a person other than the researcher who provides information (data) about the phenomenon being studied.

Low inference descriptors, which are concrete, precise, and almost literal descriptions of phenomena.Comprehensive sampling, in which all members of an entire group are selected as the units of study.Maximum variation sampling, in which objects of observation are selected because they represent disparate examples of the phenomenon being studied.

Critical case sampling, in which a unit is selected for study because it represents a dramatic example of the phenomenon being studied.Network (sometimes known as snowball) sampling, in which future successive participants are identified by prior participants.Negative case / discrepant data, in which a unit is not behaving within the parameters of the emerging pattern.

Quantitative Research

To facilitate discussion, quantitative research methods may be divided into descriptive and experimental research.Quantitative research involves deductive logic, focused and specific questions, collection and analysis of empirical data, and generalization of results.

Descriptive research does not involve manipulation of variables, whereas experimental research does involve manipulation of one or more variables.Descriptive research is conducted to provide a characterization of a situation or circumstance, or a complete account of a process.Descriptive research does not alter whatever is being studied.

Descriptive research methods include historical, case study, field study, survey, developmental, causal comparative, and correlational research.Historical research is the systematic search for facts relating to questions about the past and the interpretation of those facts.

Historical research involves specific definition of the time period addressed within the research.It encompasses use of primary and secondary sources.Primary sources are first-hand accounts or original source documents, whereas secondary sources involve interpretations of primary source data or information.

Case study research is the study of a single individual for a specified period of time.Case study research involves assessment, but not manipulation, of a variety of variables that potentially contribute to the persons current situation.Case study research also may be conducted through use of a qualitative research paradigm.

Field study research is the simultaneous study of a small number of people.Field study research is sometimes conceived of as a multiple case study, but includes consideration of the members interactions.Field study research also may be conducted through use of a qualitative research paradigm.

Survey research is the collection of data from members of a population to determine the current status of the population with respect to one or more variables.Survey research is often a precursor to other types of research, most commonly correlational or experimental research.

Survey research methods often include use of questionnaires, surveys, observations, interviews, or sociometry.Survey research typically includes self-reported data.

Developmental research is the study of one or more variables in a group of people over a relatively long period of time (i.e., long enough for potential developmental changes to occur).Developmental research is usually considered synonymous with the term longitudinal research.Developmental research may be either cross-sectional or longitudinal in nature.

Cross-sectional research involves studying cross sections (i.e., subgroups) of a population presumed to be at different developmental levels to determine if develop-mental patterns or trends exist as predicted.Longitudinal research involves studying the same group of people over a relatively long period of time to determine if developmental patterns or trends exist as predicted.

Other types of developmental research include:trend studies, in which a given general population is sampled at each data-collection point in time.cohort studies, in which a specifically-defined population is followed over time.panel studies, in which the same, presumably representative, panel (i.e., relatively small group) is assessed periodically.

Causal comparative research is usually considered synonymous with the term ex post facto research.Causal comparative research is an attempt to attribute causation without experimental manipulation of a variable.Causal comparative research is based on the premise that both the effect and the alleged cause exist at the time the research is conducted.

Correlational research includes studies having as the purpose to determine the relationships between or among variables through the use of correlational statistics.Correlational research has the advantage of allowing study of many variables simultaneously.Correlational research is probably the most common type of research in the counseling and development professions.

Experimental research is conducted to determine if differences result from the interjection of some phenomenon into peoples lives.Experimental research always involves intentional manipulation of one or more variables.

comparing conditions under various stages of the treatment (e.g., pre-post).Experimental research also involves:systematic manipulation of experimental conditions in which extraneous influences are controlled or eliminated.application of Kerlingers MAXMINCON principle.

Kerlingers MAXMINCON principle is that experimental research should:maximize the independent variables effects on the dependent variable (i.e., maximize treatment variance).minimize error factors and/or variance.control extraneous variance.

True experimental research means that the researcher (theoretically) has control over all the relevant variables.True experimental research is usually conducted in a laboratory or other highly controlled settings.True experimental research is relatively rare in the counseling and development profes-sions due to the extensive control required.

Quasi-experimental research approximates true experimental research except that complete control of all relevant variables is not possible.Quasi-experimental research is usually differentiated from true experimental research by the inability to assign subjects to groups randomly (e.g., intact groups are used) or lack of a control group.

Single-subject experimental research involves studying an individual in both treatment and nontreatment conditions and evaluating performance on the dependent variable in both conditions.Single-subject experimental research should not be confused with a case study; in single-subject research, considerable effort is given to controlling extraneous variables.

Action research is an attempt to solve a specific, immediate, and concrete problem in a local setting.Action research is not concerned with generalization to any significant degree.Action research often is used to test the effectiveness of new skills or methods.Action research often lacks general credibility due to vague definitions and controls.

Experimental Research Designs

The following are commonly used abbreviations in the depiction of experimental research designs:R = random assignment to groupsNR = nonrandom assignment to groupsE = experimental (or treatment or tx ) groupC = control groupO = observation (i.e., measurement)X = treatment (i.e., intervention)

Between Groups designs involve comparison of the variable(s) across (i.e., between) two or more tightly-controlled conditions (e.g., experimental or control).Experimental research designs can be divided into two types, depending upon the nature of the comparison or type of effect to be evaluated: between groups and within groups designs.

Because within subjects designs, such as the crossover, counter-balanced, and Latin Squares designs, are used relatively rarely, the focus here will be on the between groups designs.Within Groups designs involve each subject being exposed to each treatment condition, but under a randomly assigned sequence of treatment presentations (i.e., each subject is his/her own control).

In general, experimental research designs may be divided into three categories based on the degree of control over extraneous variance (i.e., the degree to which a difference can be attributed solely to the treatment):Pre-Experimental DesignsQuasi-Experimental DesignsTrue-Experimental Designs

One-Group Posttest-Only DesignAssign- ment Group Pre Treatment Post

NREXOPre-Experimental Designs

Nonequivalent Groups Posttest Only DesignAssign- ment Group Pre Treatment Post

NREXO NRCOPre-Experimental Designs

Nonequivalent Groups Alternate Treatment Posttest-Only DesignAssign- ment Group Pre Treatment Post

NRE1X1O NRE2X2OPre-Experimental Designs

Nonequivalent Groups Pretest-Posttest DesignAssign- ment Group Pre Treatment Post

NREOXO NRCOOQuasi-Experimental Designs

Single-Group Interrupted Time Series DesignAssign- ment Group Pre Treatment Post

NR EO1 O2 O3 O4 X O5 O6 O7 O8Quasi-Experimental Designs

Control-Group Interrupted Time Series DesignAssign- ment Group Pre Treatment Post

NR EO1 O2 O3 O4 X O5 O6 O7 O8 NR CO1 O2 O3 O4 O5 O6 O7 O8Quasi-Experimental Designs

Pretest-Posttest Control Group DesignAssign- ment Group Pre Treatment Post

REOXO RCOOTrue-Experimental Designs

Posttest-Only Control Group DesignAssign- ment Group Pre Treatment Post

REXO RCOTrue-Experimental Designs

Solomon Four-Group DesignAssign- ment Group Pre Treatment Post

RE1OXO1 RC1OO2 RE2XO3 RC2O4True-Experimental Designs

True-Experimental Designs continuedFactorial experimental designs contain more than two independent variables.In the language of factorial designs (which should not be confused with the statistical process of factor analysis), a factor is an independent variable.Each factor in a factorial design has at least two levels.

Levels in this context is synonymous with categories.Levels in this context is really a poor choice of words, because no hierarchal order of levels/categories in the variables is implied or necessary.For example, the variable gender in a factorial design has two levels (i.e., male and female), but clearly no hierarchy exists in these categories.

The notation for a factorial design is a set of numbers that identifies how many independent variables are involved and the number of levels in each of the variables.For example, an experiment in which the inde-pendent variables are group (experimental or control), gender (male or female), and secondary school grade level (9, 10, 11, or 12) would be denoted as a:

In the 2 x 2 x 4 factorial design example presented, there are three factors (independent variables) and the first factor has two levels (categories), the second factor has two levels, and the third factor has four levels.The number of numbers is the number of factors, and the numbers themselves are the number of levels in each of the respective factors.

The order of the numbers in a factorial design notation is not fixed by any rule. The example presented could have been denoted as a 2 x 4 x 2 factorial design, in which the first factor was gender, the second was grade level, and the third was group.

The (factorial experimental design) diagram for the example presented would be:

This concludes Part 1 of the presentation on RESEARCH AND PROGRAM DEVELOPMENT