Week 3 Primary Data Collection: Survey Research Observation Method Experimental Research

By Dr. Muhammad Ramzan mramzaninfo@gmail.com, 03004487844 Edited by Ahsan Khan Eco ahsankhaneco@yahoo.com 03008046243

Survey ResearchTo collect primary data, specifically for a project in hand.Surveys provide a quick, inexpensive, efficient and accurate means of assessing information about a populationIdentifying characteristics of a target market, measuring customer attitude and describing consumer purchase patterns etc.

*Survey MethodThe survey method is based on the questioning of respondents regarding their behaviour, intentions, attitudes, awareness, motivations, demographic and lifestyle characteristics. These questions may be verbally, in writing, or via computer and responses may be obtained in any of these forms. Typically the questioning is structured and a sample of respondents are asked to select from a pre-determined set of alternatives. Example: DisagreeAgreeShopping at malls is fun12345

*Types of SurveysStructured versus unstructuredstructured-formal standardised questions usedunstructured -informal, no standardised questionsdegree of structure influences choice of media

Disguised versus undisguiseddisguise - the concealing of purpose or sponsorship of a study until completionundisguised - respondent is aware of purpose and sponsor of research

*Advantages of Survey Methodsaccommodate large sample sizes; increases generalisability of resultsstandardisation all respondents react to questions worded identically; response options/scales are sameadministrative ease much simpler than a focus group/ in-depth interview; development of questionnaire is a more complex process than the administrationability of tapping into factors & relationships that are not directly observable ie attitudes, feelings, preferencestabulation and statistical analysis of data is easysubgroup differences can be easily determined

*Disadvantage of Survey Methodsdifficulty of developing accurate survey instruments (questionnaire design)limits to the in-depth detail of data lack of control over timeliness, & potential low response ratesdifficulties to determine if respondents are responding truthfullymisinterpretation of data results & inappropriate use of data analysis procedures

A Classification of Survey Methods-Mode of Administration

*Mail MethodsCan be conducted through mailing questionnaires to pre-selected respondents via ordinary mail or mail panel. It includes outgoing envelope, cover letter, questionnaire, return envelope, the purpose, assurance of anonymity, an appeal for completion and return within due date. There is no verbal interaction between researcher and the respondentBefore stat of mail survey, respondents need to be broadly identified, a valid mailing list, closely related to population of interest is required

Mail Survey Advantages & Disadvantagesno interviewer bias, can respond at leisurehigh geographic flexibility, more confidential informationcost confined to mailing list, forms and postagefollow up easy but time consuming

low response rate, high cost per response, long time delays, no probing/observation; can read entire questionnaire before answering which may influence response; higher possibility misunderstand/clarificationnever sure that the target respondent actually answered the questionscomplex questions not responded to well; open-ended questions do not achieve lengthy written responses

Increasing Response Rates for Mail Surveys Effective cover letterAuthorities help/reference letter Incentives helps Interesting questionsUse of peers/colleagues/professional association Follow-ups Advanced notification Survey sponsorship Keying questionnaires

Personal MethodsPersonal In-home Interviews; respondents are interviewed face-to-face in their homes. Interviewer contact respondents ask questions and record responses

Mall-Intercept: Respondents are intercepted while they are shopping in malls/superstores and brought to test facilities in the malls. Researcher administers a questionnaire. Methods is more efficient and becoming popular

Computer Assisted Personal Interviewing: Respondent sits in front of a computer terminal and answer questions on the computer screen using mouse and keyboard. User-friendly electronic packages are available to design easily understandable questions. Interviewer remains available as a host to guide the respondent

*Personal Methods Advantages & Disadvantagespersuasion/cooperation; better response rate; use observationcan use visual material; longer questionnairesCan assist to interviewee; more precise selection of samplemore versatile questioning; extended probing possibleif self-completed, respondent can complete when convenient

travel time & expenses high, people reluctant to talk to strangersinterviewers presence may bias, inhibited, difficult to supervise and control; difficult to recruit capable interviewers; geographic flexibility limitedcall back/follow up difficult; no anonymity of intervieweesSelf-completed disadvantages similar to mail

Telephone MethodsTraditional Telephone Interview involves phoning and asking a series of questions to respondents. Researcher uses paper questionnaire and records responses with a pencil

Computer-Assisted Telephone Interviewing: Researcher wears a mini-hand set, reads questions from compute screen and records responses directly in the computer memory. In this method computer replaces pencil and paper questionnaire

Telephone Method Advantages & Disadvantagesdialling efficient especially call backs; travel avoidedrespondent does not have to open door to strangersgeographic coverage good; supervision and training excellent; speed of data collectionless interviewer bias; greater anonymity of respondents

no observation; visuals excluded; retaining attention more difficult therefore shorter questionnaires; only homes with telephones; cost can be a factor with STD callsthere is limited interview times pertaining to the best times for making and gaining responsesInterviewer is unable to control the interview; the respondent can hang up at any time

Electronic MethodsEmail Interviews require list of email addresses. Survey is written within the body of email message. Respondents type answers to close ended/open ended questions at designated places and reply through email. It requires data entry.

Internet Interviews are posted on a website using HTML. Respondents need to open the website and complete the survey. Respondents are sent links usually through email. (Survey Monkey, Google Doc Forms)

*Electronic Methods Advantages & Disadvantagesspeed of response/ data collectioneasy of development and testinglittle cost of administrationeasy to administer

the possibility of data corruption via virus transmissionpossible unreliability of e-listsare respondents representative of the populationpossibility of bogus replies

CriteriaPhoneIn-Home InterviewsMall-Intercept InterviewsCAPIMail SurveysMail PanelsE-MailInternetA Comparative Evaluation of Survey Methods

Flexibility of data collection

Moderate to high

High

High

Moderate to high

Low

Low

Low

Moderate to high

Diversity of questions

Low

High

High

High

Moderate

Moderate

Moderate

Moderate to high

Use of physical stimuli

Low

Moderate to high

High

High

Moderate

Moderate

Low

Moderate

Sample control

Moderate to high

Potentially high

Moderate

Moderate

Low

Moderate to high

Low

Low to moderate

Control of data collection environment

Moderate

Moderate to high

High

High

Low

Low

Low

Low

Control of field force

Moderate

Low

Moderate

Moderate

High

High

High

High

Quantity of data

Low

High

Moderate

Moderate

Moderate

High

Moderate

Moderate

Response rate

Moderate

High

High

High

Low

Moderate

Low

Very Low

Perceived anonymity of the respondent

Moderate

Low

Low

Low

High

High

Moderate

High

Social desirability

Moderate

High

High

Moderate to High

Low

Low

Moderate

Low

Obtaining sensitive information

High

Low

Low

Low to moderate

High

Moderate to High

Moderate

High

Potential for interviewer bias

Moderate

High

High

Low

None

None

None

None

Speed

High

Moderate

Moderate to high

Moderate to high

Low

Low to moderate

High

Very high

Cost

Moderate

High

Moderate to high

Moderate to high

Low

Low to moderate

Low

Low

*Factors determining choice of survey methodResearchers resources & objectives Time horizonBudget Desired quality of data Required level of generalizationCompleteness & precision

Respondent characteristicsWillingness to participateAbility to participateDiversity of respondents

Type of questions askedComplexity Topic sensitivityAmount of information required per respondent

There is no best form of survey; each has advantages and disadvantages.

Selected Questions to Determine the Appropriate Technique Is the assistance of an interviewer necessary? Are respondents interested in the issues being investigated? Will cooperation be easily attained? How quickly is the information needed?Will the study require a long and complex questionnaire?How much is the budget?

Observation MethodIt involves recording the behavioral pattern of people, objects, and events in a systematic manner to obtain information about a phenomenon of interest.

The observer does not question or communicate with the people being observed.

Information obtained may be structured or unstructured, direct or indirect and may be conducted in a natural or contrived environment

Structured vs Unstructured ObservationFor structured observation, the researcher specifies in detail what is to be observed and how the measurements are to be recorded, e.g., an auditor performing inventory analysis in a store.

In unstructured observation, the observer monitors all aspects of the phenomenon that seem relevant to the problem at hand, e.g., observing children playing with new toys.

Disguised vs Undisguised Observation

In disguised observation, the respondents are unaware that they are being observed. Disguise may be accomplished by using one-way mirrors, hidden cameras, or unnoticeable mechanical devices. Observers may be disguised as shoppers or sales clerks.

In undisguised observation, the respondents are aware that they are under observation.

Natural vs Contrived ObservationNatural observation involves observing behavior as it takes places in the environment. For example, one could observe the behavior of respondents eating fast food in Burger King.

In contrived (artificial) observation, respondents' behavior is observed in an artificial environment, such as a test kitchen.

A Classification of Observation Methods

Personal ObservationA researcher observes actual behavior as it occurs. The observer does not attempt to manipulate the phenomenon being observed but merely records what takes place. For example, a researcher might record traffic counts and observe traffic flows in a department store.

Mechanical Observation Mechanical devices rather than human observers record the phenomenon.Do not require respondents' direct participation.Audimeter (attached to a television to continuously record what channel that set is tuned. People meters attempted to measure the channel to which TV is tuned but also who is watching)Turnstiles records number of people entering or leaving a building.On-site cameras (still, motion picture, or video) Optical scanners in supermarketsDo require respondent involvement. eye-tracking monitors; psychogalvanometers (skin response)voice pitch analyzers (emotional reactions through changes in respondents voicedevices measuring response latency (amount of time it takes to respond

Observation Methods: AuditThe researcher collects data by examining physical records or performing inventory analysis. Data are collected personally by the researcher. The data are based upon counts, usually of physical objects. Retail and wholesale audits conducted by marketing research suppliers

Observation Methods: Content AnalysisThe objective, systematic, and quantitative description of the manifest content of a communication. The unit of analysis may be words, characters (individuals or objects), themes (propositions), space and time measures (length or duration of the message), or topics (subject of the message). Analytical categories for classifying the units are developed and the communication is broken down according to prescribed rules.

Observation Methods: Trace AnalysisData collection is based on physical traces, or evidence, of past behavior.

The number of different fingerprints on a page was used to gauge the readership of various advertisements in a magazine.The position of the radio dials in cars brought in for service was used to estimate share of listening audience of various radio stations. The age and condition of cars in a parking lot were used to assess the affluence of customers.The magazines people donated to charity were used to determine people's favorite magazines.Internet visitors leave traces which can be analyzed to examine browsing and usage behavior by using cookies.

A Comparative Evaluation of Observation MethodsCriteria Personal Mechanical Audit Content Trace Observation Observation Analysis Analysis Analysis

Degree of structureLowLow to highHigh High MediumDegree of disguise MediumLow to highLow High High Ability to observe HighLow to highHigh MediumLow in natural setting Observation bias HighLowLow Medium MediumAnalysis Bias HighLow toLow Low MediumMedium General remarks Most Can be Expensive Limited to Method offlexible intrusive commu- last resortnications

Relative Advantages of Observation They permit measurement of actual behavior rather than reports of intended or preferred behavior. There is no reporting bias, and potential bias caused by the interviewer and the interviewing process is eliminated or reduced.

Certain types of data can be collected only by observation. If the observed phenomenon occurs frequently or is of short duration, observational methods may be cheaper and faster than survey methods.

Relative Disadvantages of Observation The reasons for the observed behavior may not be determined since little is known about the underlying motives, beliefs, attitudes, and preferences. Selective perception (bias in the researcher's perception) can bias the data. Observational data are often time-consuming and expensive, and it is difficult to observe certain forms of behavior. In some cases, the use of observational methods may be unethical, as in observing people without their knowledge or consent.

It is best to view observation as a complement to survey methods, rather than as being in competition with them.

Experiment MethodA research investigation in which conditions are controlled so that causal relationship among variables is determinedOne or more independent variable are manipulated while holding all other possible independent variables to observe the effects on dependent variableIVs effect on a dependent variable is measuredUsed to test a hypothesis

Use of experimental researchExperimentation is commonly used to infer causal relationshipCausality is the relationship between an event (the cause) and a second event (the effect), where the second event is understood as a consequence of the firstA. the relationship of cause and effect B. the principle that nothing can happen without being caused C. causal agency or quality

Basic issues of experimental designManipulation of the independent variableSelection of dependent variableAssignment of subjects (or other test units)Control over extraneous variablesThe experimenter has some degree of control over the independent variable.

The variable is independent because its value can be manipulated by the experimenter to whatever he or she wishes it to be

Definitions and ConceptsIndependent variables are variables or alternatives that are manipulated and whose effects are measured and compared, e.g., price levels. Test units are individuals, organizations, or other entities whose response to the independent variables or treatments is being examined, e.g., consumers or stores. Dependent variables are the variables which measure the effect of the independent variables on the test units, e.g., sales, profits, and market shares. Extraneous variables are all variables other than the independent variables that affect the response of the test units, e.g., store size, store location, and competitive effort.

Independent Variable (IV)The experimenter controls independent variable.The variables value can be manipulated by the experimenters to whatever they wish it to be.

Dependent Variable (DV)Its value is expected to be dependent on the experimenters manipulationCriterion or standard by which the results are judgedSelectione.g... sales volume, awareness, recall,MeasurementDVs are the outcome of interest to the researcher and the decision maker

Example

Hypothesis: Changes in price influences salesPrice here is IV and sales DV and other possible variables are extraneous variables

ExampleInfluence of brand name (IV) on consumers taste(DV) perceptionsExperimenter manipulated whether consumers preferred drinks of labeled or unlabelled bottlesOne week respondents were given drink a pack of bottles with label ABC. Next week respondents received another pack of drinks with brand labels (Pepsi, Fanta, Shezan etc)Researcher measured reactions to drinks after each tasting as the drink was the same.Therefore the differences observed in taste (DV) could only be attributed to the difference in labeling (IV). Participants preferred for branded drinks, so the conclusion (inference) is that brand name does influence consumers taste perception

How may an experimenter control forextraneous variation?Eliminate extraneous variablesHold conditions constantRandomizationMatching subjects

Experimental DesignAn experimental design is a set of procedures specifying:the test units and how these units are to be divided into homogeneous (all the same) subsamples, what independent variables or treatments are to be manipulated, what dependent variables are to be measured; and how the extraneous variables are to be controlled.

Validity in ExperimentationA researcher has to draw valid conclusions of effects of independent variables (internal validity) on the study group (sample) and make valid generalization (external validity) on larger population of interestInternal validity refers to whether the manipulation of the independent variables or treatments actually caused the observed effects on the dependent variables. Control of extraneous variables is a necessary condition for establishing internal validity.External validity refers to whether the cause-and-effect relationships found in the experiment can be generalized. To what populations, settings, times, independent variables and dependent variables can the results be projected?

Extraneous variablesHistory refers to specific events that are external to the experiment but occur at the same time as the experiment.Maturation (MA) refers to changes in the test units (sample) themselves that occur with the passage of time. Testing effects are caused by the process of experimentation. Typically, these are the effects on the experiment of taking a measure on the dependent variable before and after the presentation of the treatment. The main testing effect (MT) occurs when a prior observation affects a latter observation.

Extraneous VariablesIn the interactive testing effect (IT), a prior measurement affects the test unit's response to the independent variable. Instrumentation (I) refers to changes in the measuring instrument, in the observers or in the scores themselves. Statistical regression effects (SR) occur when test units with extreme scores move closer to the average score during the course of the experiment. Selection bias (SB) refers to the improper assignment of test units to treatment conditions. Mortality (MO) refers to the loss of test units while the experiment is in progress.

Controlling extraneous variablesRandomization refers to the random assignment of test units to experimental groups by using random numbers. Treatment conditions are also randomly assigned to experimental groups.Matching involves comparing test units on a set of key background variables before assigning them to the treatment conditions. Statistical control involves measuring the extraneous variables and adjusting for their effects through statistical analysis.Design control involves the use of experiments designed to control specific extraneous variables.

Laboratory VS Field experimentsA laboratory environment is an artificial one, which the researcher constructs with the desired conditions specific to the experiment. An artificial environment of experimentation in which the researcher constructs the desired conditions. To measure the effectiveness of a test commercial could be conducted in a laboratory environment by showing the test commercial embedded in a TV program to respondents in a test theater.

Field environment is an experiential location set in actual market conditions. Such as running test commercials on actual TV stations.

Laboratory vs field experiments

Laboratory experimentField experimentArtificial-Low RealismFew ExtraneousVariablesHigh controlLow CostShort DurationSubjects Aware ofParticipationNatural-High RealismMany ExtraneousVariablesLow controlHigh CostLong DurationSubjects Unaware ofParticipation

Limitations of experimentationExperiments can be time consuming, particularly if the researcher is interested in measuring the long-term effects.Experiments are often expensive. The requirements of experimental group, control group, and multiple measurements significantly add to the cost of research.Experiments can be difficult to administer. It may be impossible to control for the effects of the extraneous variables, particularly in a field environment. Competitors may deliberately contaminate the results of a field experiment.

Test marketingIt is an application of a controlled experiment done in limited but carefully selected test market. It involves a replication of the planned national marketing program for a product in the test markets. Often market mix (independent) variables are varied in test marketing and the sales (dependent variable) are monitored so that an appropriate national marketing strategy can be identified

To test market acceptance of the productTest alternative levels of marketing mix variables.

Criteria for the selection of test marketsBe large enough to produce meaningful projections. They should contain at least 2% of the potential actual population.Be representative demographically and with respect to product consumption behavior.Be representative with respect to media usage and with respect to competition.Be relatively isolated in terms of media and physical distribution.Have normal historical development in the product class.Have marketing research and auditing services available.Not be over-tested.

SummaryThe scientific notion of causality implies that we can never prove that X causes YWe can only infer that X is one of the causes of Y in that it makes the occurrence of Y probable's.Three conditions must be made (I) concomitant, which implies that X and Y must vary together in a hypothesized way (II) time order of occurrence of variables, which implies that X must precede Y and (III) elimination of other possible causal factors. Experiments provide the most convincing evidence of these conditions.An experiment is formed when one or more independent variables are manipulated by the researcher and their effect on one or more dependent variables is measured.

Conclusion

There is no best data collection technique; each has its advantages and disadvantages.

The suitability and appropriateness of data collection technique/method depends on the purpose, scope of the research and researchers resources.

THANK YOUBy Dr. Muhammad Ramzan mramzaninfo@gmail.com, 03004487844 Edited by Ahsan Khan Eco ahsankhaneco@yahoo.com 03008046243

The Superior University******************************************