New to Online Surveys? Try It, You’ll Like It

A N S W E R S E A R C H, I N C.

GreenBook.org shares market research articles, white papers, case studies and presentations published by market research companies.

Follow us @ResearchShare.

1

New to Online Surveys? Try It, You’ll Like ItAn overview of the fastest growing market research methodology--online surveys: Compares online research to traditional methodologies and provides important tips for maximizing the success of online survey projects.

SUMMARY AND CONCLUSIONS

Online research is a widely used methodology. The acceptance of online research as a marketing tool is evidenced by the huge growth in the use of this methodology by marketing professionals.

Online research is a proven technique. Online surveys result in the same conclusions and marketing decisions as do surveys using phone, mall, or mail interviewing.

All survey methodologies are imperfect. Online research shares several of the problems of other methods, but in many respects online research can be the best choice. It is quicker and costs less than others. It also offers a better environment for the respondent—it is non-intrusive, available 24/7, and can be done on the respondent’s schedule. It provides greater privacy and anonymity to the respondent.

Response rates for online surveys tend to be low. However, response rates for phone, mall, and mail surveys are also quite low. Online research usually obtains response rates in the 2 to 3% range, but response rates can be much higher (70% or better) depending upon the sample used.

Other research has shown that the pattern of non-response (i.e., which population groups are least likely to respond) is similar for internet studies and studies using other methodologies. The proper invitation, incentives, and questionnaire design can minimize non-response bias. Data management can also compensate for non-response bias.

Online surveys are easy to implement, but an experience market research professional is just as important with online studies as with other types of market research studies.

DETAILED FINDINGS

Online research is a proven, trusted, and widely used methodology.

Phone: 407-677-5939 8745 Belter Drive www.answersearchinc.comFax: 407-677-4991 Orlando FL 32817 AnswerSearch@earthlink.net

A N S W E R S E A R C H, I N C.

2

The use of online research to replace traditional research methods has been growing. According to Inside Research (January 2006), almost one third of all spending for surveys is for online surveys.

Inside Research reported that spending for online research has grown every year since its inception.

Online research has been proven to be valid.

Research shows the online data collection almost always leads to the same business decisions as traditional data collection methods.

Online research is suitable for most, but not all, research projects.

The main criteria for determining if a survey is appropriate for the internet are:

-- The survey is suitable for a self-administered questionnaire.

-- The target audience is reachable via email.

-- The population with access to the Internet reflects the client’s audience.

-- Open-end questions are minimal, since there is no interview present to probe for in-depth answers.

Online research has many benefits over traditional methodologies.

The obvious advantages of online research are quick turnaround and low cost. However there are many other advantages.

Online research is respondent-driven. Respondents complete the survey at their leisure on their own schedule. They can choose a time when they are not distracted. Additionally, online research does not engender the negativity that can arise from the intrusiveness of phone and mall solicitations for cooperation.

Respondents may be more cooperative and more honest because they have greater anonymity for questions dealing with sensitive topics.

Visuals can be embedded in the questionnaire or a link can be inserted at the appropriate points, putting online research on par with personal interviews with respect to the ability to use audio-visual stimuli.

The response rate for online research studies is often superior to that currently obtained with traditional methodologies.

3

The research industry has suffered from declining response rates for years. CMOR (The Council for Marketing and Opinion Research) reported 12% response rates in 2002, down from 16% in 2000 for telephone and mall studies. In contrast, online research sometimes reaches a 70% response rate.

Incentives can improve response rates. A cash incentive is the most effective motivator. AnswerSearch, Inc. has had success with using “sweepstakes” and with using electronic $10 Amazon.com gift cards offered to each respondent who completes a questionnaire.

Additionally, response rates can be improved by the wording of the invitation, particularly the subject line. The subject line should entice to recipient to read further. Additionally, a sponsor that is familiar to the respondents improves response rates. And, of course, care should be taken to avoid using words or phrases that are likely to set off spam blockers.

A brief questionnaire with simple straightforward questions will improve response rates.

The non-response bias with internet research is no different than that found with traditional methodologies.

Certain population groups have historically been less receptive to cooperate with surveys. Research conducted by e-Rewards (a company providing email panels) shows that the demographic patterns of non-response are the same for internet research as they are for traditional research methodologies.

The groups with below average response rates are older people, less educated people, lower income people, and sometimes males.

However, non-response bias is not always present. If invitations are sent to a representative sample, the probability of a balanced response is high. AnswerSearch, Inc., a full service market research company, recently conducted a mail study done with a database sample and obtained a 13% response rate. Because it was a database sample, demographic and behavioral information was available for the entire outgoing sample. There were either no differences or only very small differences between responders and non-responders on 12 demographic and behavioral characteristics.

If non-response bias is suspected, there are ways to minimize this bias. The outgoing sample and the incoming responses can be managed to achieve representativeness by setting requirements for eligibility and quotas for demographic and behavioral characteristics. Data can also be weighted to match known parameters.

Online interviewing allows for the use of various types of samples.

Panel Sample-- People who have been pre-recruited to participate in research projects. -- Panelists provide demographic and product usage, etc.-- High response rates; usually from 15% to 70%

Database Sample-- People, usually customers, who have become part of a company database

4

-- A variety of information about the database members is available-- Response rates are usually in the 5% to 15% range

Push-Sample (Lists) -- People who have agreed to receive email messages related to a topic of interest, but not

necessarily to participate in research-- Respondents are “pushed” to the survey site by responding to an email invitation -- No information on respondents available-- Usually low response rates—1% to 2%

Pull-Sample-- People are “pulled” to the research site by banners and ads-- Little is known about them except the information they may provide when they respond

Quilt Sample-- Uses a mix of the above four types of samples

A final word of caution

The apparent ease and simplicity of online surveys may encourage a DIY (Do It Yourself) mindset. This would be a mistake. The same expertise that goes into study design, questionnaire design, and analysis for traditional methodologies is required for online surveys as well. It is still important to use an experienced market research professional for online research in order to insure valid actionable results.

5