THE SURVEY METHOD IN RESEARCH

BY:

CHRISTINE HABERSHAM,

BRITTANY LOHMEIER , &

BILLIE RAMES-SCHULTZ

UNIVERSITY OF NEW MEXICO

A BASIC DEFINITION OF A SURVEY

“SURVEYS PRODUCE INFORMATION TO DESCRIBE, COMPARE, AND PREDICT ATTITUDES, OPINIONS, VALUES, AND BEHAVIOR BASED ON WHAT PEOPLE SAY OR SEE AND WHAT IS CONTAINED IN RECORDS ABOUT THEM AND THEIR ACTIVITIES” (FINK, 1995, P. 14)

WHO USES SURVEYS?

• GOVERNMENT(I.E. TO FIGURE OUT HOW MANY REPRESENTATIVES)

• GOVERNMENT AGENCIES (I.E. TO FIGURE OUT THE EFFECTIVENESS OF THE SOCIAL WELFARE PROGRAMS THROUGH THE DEPARTMENT OF WELFARE)

• UNIVERSITIES, BOTH NATIONWIDE AND STATEWIDE

• PRIVATE SECTOR LIKE BUSINESSES OR THE MEDIA

(ROSSI, WRIGHT, & ANDERSON, 1983)

OBJECTIVES

• THE OBJECTIVES ARE SPECIFICALLY DESIGNED TO BE MEASURABLE IN ORDER TO DESCRIBE, COMPARE, OR PREDICT THE RELATIONS BETWEEN TWO VARIABLES. THESE TWO VARIABLES ARE THE INDEPENDENT AND DEPENDENT VARIABLES.

VARIABLES

• INDEPENDENT VARIABLES: ARE VARIABLES THAT CAN STAND ALONE. THEY CHANGE NO MATTER WHAT HAPPENS.

• DEPENDENT VARIABLES: THESE VARIABLES RELY ON THE INDEPENDENT (OR OTHER) VARIABLES IN ORDER TO CHANGE.

• EXAMPLE: HOW FAR YOU DRIVE (INDEPENDENT VARIABLE) IS HOW MUCH GAS YOU USE (DEPENDENT VARIABLE).

DEFINITIONS• REPRESENTATIVE SAMPLING: PART OF THE TARGETED POPULATION IS ASKED TO PARTICIPATE

IN THE SURVEY (FINK, 1995).

• GENERALIZABILITY: IT IS WHEN THE SAMPLE GROUP DOES REPRESENT THE TARGET POPULATION AND THE RESULTS CAN BE USED IN THAT TARGET POPULATION ((FINK, 1995).

• MEASUREMENT: IS A WAY TO ASSIGN NUMBERS SO THAT THEY CAN BE ANALYZED THROUGH CERTAIN RULES (BOHRNSTEDT, 1983).

• TWO TYPES OF RESEARCH DESIGN:

• CROSS-SECTIONAL STUDIES: DONE AT ONE CERTAIN POINT IN TIME, BUT DO NOT PROVIDE INFORMATION ON CAUSATION.

• LONGITUDINAL STUDIES: THEY ARE DONE AT MULTIPLE POINTS OF TIME THAT CAN STUDY THE SAME PARTICIPANTS, CHANGES IN THE GENERAL POPULATION, OR LOOK AT A CERTAIN GROUP OR AGE.

(GRIMM & WOZNAIK, 1990)

DEFINITIONS (CONT.)

• TYPES OF SURVEYS:

• QUESTIONNAIRES: SET OF DEVELOPED QUESITONS THAT ARE NOT CREATED ON THE SPOT.

• SCHEDULES: USED FOR OBSERVATIONS AND USE THE SAME BASIC GUIDELINES AS THE QUESTIONNAIRES.

• INTERVIEWS: DONE FACE-TO-FACE OR OVER THE PHONE AND NORMALLY DONE QUICKLY.

(YOUNG, 1996)

DESIGN

• DEPENDING ON WHAT THE RESEARCHERS WANT TO GET OUT OF A SURVEY SHOULD LEAD THE RESEARCHERS INTO DECIDING WHETHER TO CONDUCT A CROSS-SECTIONAL OR A LONGITUDINAL STUDY.

• THEN THIS SHOULD LEAD THE RESEARCHERS IN DECIDING WHAT TYPE OF SURVEY TO USE: QUESTIONNAIRE, SCHEDULE, OR INTERVIEW.

• THIS SHOULD THEN LEAD THE RESEARCHERS INTO WHAT TYPE OF QUESTIONS TO USE: OPEN-ENDED OR CLOSED-ENDED.

QUESTIONS

• CLOSE-ENDED QUESTIONS: QUESTIONS THAT HAVE BEEN WRITTEN WITH A SET OF ANSWER AND DOES NOT VIE THE PARTICIPANTS THE FREEDOM TO ADD ADDITIONAL INFORMATION. THESE QUESTIONS ARE TYPICALLY USED IN QUESTIONNAIRES OR SCHEDULES.

• OPEN-ENDED QUESTIONS: QUESTIONS THAT DO NOT HAVE SPECIFIED ANSWERS AND ALLOW RESEARCHERS TO GAIN MORE INFORMATION FROM THE PARTICIPANTS. THESE QUESTIONS TEND TO BE USED MORE DURING INTERVIEWS OR AT THE END OR QUESTIONNAIRES OR SCHEDULES.

(PETERSON, 2000)

IMPLEMENTATION

• SHOULD BE STANDARDIZED, DONE IN A RULE-GOVERNED AND ROUTINE FASHION, SO THE RESULTS ARE GENERALIZABLE.

PILOT TEST

• TO HAVE A SAMPLE GROUP PARTICIPATE IN A SURVEY TO SEE WHAT ERRORS COULD EXIST IN THE SURVEY BEFORE CONDUCTING THE ACTUAL SURVEY.

MEASUREMENT OF SURVEY

• THERE ARE CERTAIN RULES THAT NEED TO BE FOLLOWED WHEN DESIGNING RATING SCALES AND GIVING NUMERICAL VALUES(CODING) IN ORDER TO ANALYZE THE RESPONSES WHEN THE SURVEYS HAVE BEEN COMPLETED.

• THE RESEARCH HAS TO BE RELIABLE AND VALID IN ORDER TO BE CONSIDERED GOOD RESEARCH.

RELIABILITY AND VALIDITY

• RELIABILITY: IN ORDER FOR RESEARCH TO BE CONSIDERED TO BE RELIABLE IT HAS TO REMAIN CONSISTENT WITH HOW DATA IS GATHERED, INFORMANT RESPONSES, AND HOW IT IS REPORTED.

• VALIDITY: IS WHEN THE RESEARCH IS CONSIDERED TO BE RELIABLE AND ACCURATE.

• RESEARCH DOES NOT NEED TO BE NEED TO BE VALID TO BE RELIABLE; HOWEVER, IT DOES NEED TO BE RELIABLE TO BE CONSIDERED VALID.

(FINK, 1995)

ANALYZING AND REPORTING THE DATA

• RESEARCHERS COLLECT THE CODED DATA AND INSERT IT INTO A DATA SHEET IN ORDER TO COMPARE AND CONTRAST THE DATA WITHIN THE SET OF SURVEYS.

• WHEN THE RESEARCHERS REPORT THEIR DATA IT SHOULD BE CONSIDERED RELIABLE. THEN IF IT IS CONSIDERED RELIABLE, THE RESEARCHERS SHOULD STATE IF THE RESEARCH IS CONSIDERED TO BE VALID.

LIMITATIONS

1. COVERAGE (I.E. WERE ALL OF THE VARIABLES CONSIDERED)

2. SAMPLING (I.E. WAS THE SAMPLE A TRUE REPRESENTATION OF THE TARGET POPULATION)

3. NONRESPONSE (I.E. WERE THERE PARTICIPANTS THAT DID NOT COMPLETE THE SURVEY)

4. MEASUREMENT (I.E. WERE THE MEASUREMENTS INCONSISTENT WITH THE PARTICIPANTS RESULTS)

5. FINANCES (I.E. WAS THERE ONLY ENOUGH MONEY TO CONDUCT QUESTIONNAIRES WHEN THE RESEARCHERS REALLY WERE AIMING AT INTERVIEWS)

6. QUESTIONS (I.E. WERE THE QUESTIONS LEADING OR THREATENING)

(MILLER, 1983)

CRITERIA FOR A GOOD SURVEY• DOES THE SURVEY HAVE MEASUREABLE OBJECTIVE?

• ARE THERE SPECIFIC CRITERION TO MEASURE THE OBJECTIVES?

• DO THE OBJECTIVES PERTAIN OR HAVE SIGNIFICANCE IN ORDER TO BE STUDIED?

• IS THE RESEARCH DESIGN SOUND?

• WAS THE RESEARCH DESIGN STANDARDIZED?

• DID THE RESEARCHERS CONSIDER MULTIPLE VARIABLES?

• WERE THE QUESTIONS DEVELOPED IN SUCH A WAY THAT THEY WERE NOT LEADING OR THREATENING?

• ARE THE QUESTIONS HAVE MULTIPLE COMPONENTS IN ORDER TO SEEK OUT DIFFERENT VARIABLES THAT MAY BE A CAUSAL AGENT?

• WERE THE QUESTIONNAIRES, SURVEYS, OR INTERVIEWS DESIGNED IN A PROGRESSIVE FASHION THAT ALLOW THE RESPONDENT TO FEEL COMFORTABLE ANSWERING THE QUESTIONS?

• DID THE RESEARCHERS FOLLOW THE PROPER CONFIDENTIALITY PROCEDURES?

• WERE THE PARTICIPANTS AWARE THAT THEIR RESPONSES WERE INCLUDED IN A STUDY?

CRITERIA FOR A GOOD SURVEY (CONT.)

• ARE THE CHOICES IN THE POPULATION SAMPLE SOUND?

• IS THE SAMPLE GENERALIZABLE TO THE REST OF THE SPECIFIED POPULATION?

• WAS THE POPULATION SAMPLE RANDOMLY SELECTED?

• ARE THE INSTRUMENTS USED TO COMPLETE THE SURVEY RELIABLE AND VALID?

• WERE THE PROCEDURES CONSISTENT IN GATHERING AND REPORTING DATA?

• WAS THE ANALYSIS AND REPORTING OF DATA ACCURATE?

• WAS THE ANALYSIS APPROPRIATE?

• WAS THE WAY IN WHICH THE ANALYSIS COMPLETED STATED?

• DOES THE ANALYSIS MAKE SENSE?

• WERE THE RESULTS ACCURATELY REPORTED?

• DID THE RESULTS MATCH WHAT THE ANALYSIS OF DATA GATHERED?

• DID THE RESULTS MAKE SENSE?

REFERENCES

BOHRNSTEDT, G. W. (1983). MEASUREMENT. IN P. H. ROSSI, J. D. WRIGHT, & A. B. ANDERSON (EDS.) HANDBOOK OF SURVEY RESEARCH (P. 1-20). SAN DIEGO, CA: ACADEMIC PRESS.

FINK, A. (1995). HOW TO DESIGN SURVEYS. THOUSAND OAKS, CA: SAGE PUBLICATIONS.

GRIMM, J. W., & WOZNIAK, P. R. (1990). BASIC SOCIAL STATISTICS AND QUANTITATIVE RESEARCH METHODS: A COMPUTER-ASSISTED INTRODUCTION. BELMONT, CA: WADSWORTH

MILLER, W. L., (1983). THE SURVEY METHOD IN SOCIAL AND POLITICAL SCIENCES: ACHIEVEMENTS, FAILURES, PROSPECTS. LONDON, GREAT BRITAIN: FRANCES PRINTER.

PETERSON, R. A., (2000). CONSTRUCTING EFFECTIVE QUESTIONNAIRES. THOUSAND OAKS, CA: SAGE PUBLICATIONS.

ROSSI, P. H., WRIGHT, J. D., & ANDERSON, A. B., 1993. SAMPLE SURVEYS: HISTORY, CURRENT PRACTICE, AND FUTURE PROSPECTS. IN P. H. ROSSI, J. D. WRIGHT, & A. B. ANDERSON (EDS.) HANDBOOK OF SURVEY RESEARCH (P. 1-20). SAN DIEGO, CA: ACADEMIC PRESS.

YOUNG, P. V. (1966). SCIENTIFIC SOCIAL SURVEY AND RESEARCH (4TH ED.). ENGLEWOOD CLIFFS, NJ: PRENTICE- HALL.