View
23
Download
0
Category
Tags:
Preview:
DESCRIPTION
Using the web to collect data for studies in Psychology Prof. Ben C Jones. Advantages of web-based testing. Advantages: Low cost. Collecting data in the lab can be very expensive, in terms of time, money and other resources. By contrast, once set up, testing online is very cheap. - PowerPoint PPT Presentation
Citation preview
Using the web to collect data for studies in
Psychology
Prof. Ben C Jones
Advantages of web-based testing
Advantages: Low cost
Collecting data in the lab can be very expensive, in terms of time, money and other resources
By contrast, once set up, testing online is very cheap
You have to employ fewer people to collect the same volume of data
You can collect a large volume of data very quickly
Advantages: Recruitment
Some groups of participants are hard to get to come in to the lab because they are rarely on campus
By contrast, it is often easy to get these types of people to participate in an online test
People can take part in the test in their own time (e.g. at times that might not be convenient for the
experimenters) and in the setting most convenient for them (e.g. their own home)
Advantages: Large samples
A challenge for lab-based studies is collecting the large samples required for some types of research
By contrast, collecting very large samples for an online test is easier (and cheaper and faster)
It is possible to leave a study running online for months, years and, potentially, decades in order to
collect very large samples
To tie up a lab’s resources over the same period of time would be a big gamble
Advantages: Diverse samplesA problem for some areas of psychology is that we now know a great deal about how undergraduate psychology students behave, but little about how other groups might
behave
The web allows a wider range of subjects to be included in studies (that may be more representative of the general
population than you are!)
For example, the web can be used to easily and cheaply test people from different countries and/or people of different
ages, occupations, educational backgrounds, socioeconomic statuses etc.
As web access improves, this strength of web-based research will become even more important
Advantages: Pilot testing
Web-based tests are well suited to pilot tests that you can then run in the lab
This allows lab tests to target successful projects and ensure lab resources are allocated wisely
Additionally, web tests are very useful for quickly and easily piloting, e.g., new sets of stimuli, questionnaires etc.
Advantages: Public engagement
The interactive nature of web tests allows people to learn about your research and lends itself to interactive exhibits
We have adapted our web tests for interactive public exhibitions as part of French Science Week, The Royal Society of London’s Summer Science Exhibition, The
London Science Museum and The Natural History Museum
The nature of web tests allows you to reach beyond an academic audience. My lab’s website was included in the NY Times’ list of the Top 5 Psychology Websites and has
had ~3 million visitors in the last 30 months alone
Disadvantages of web-based testing
Disadvantages: Control
Some research areas require very tight control over, e.g., viewing distances, lighting etc.
This is often not possible in web tests
Consequently, web testing is better-suited to some research areas than others
Disadvantages: Reliability There are problems (potentially) with ‘frivolous’ responses,
duplicate participation etc.
These can introduce additional noise in the data sets or even bias the results
These problems can often be addressed by ‘catch trials’ and deleting data from duplicate IP addresses
Many areas have now published papers demonstrating that online data and lab data show the same effects
For some types of data (personal information etc.) there is some evidence that the greater feeling of anonymity in online tests makes responses more (not less) reliable
Disadvantages: Sampling Bias
Although web access is very good in many countries and improving all the time, some groups (e.g., very low SES)
may be underrepresented
It used to be said that online samples were biased towards teenage, single males and against women and the elderly
This is no longer the case, however (e.g., ‘silver surfers’)
Despite possible sampling biases, it is unlikely that samples are as biased as samples of UG Psychology students, for
example
Disadvantages: Expertise
Setting up web tests is not straightforward and requires great investment in, e.g., equipment (server) and expertise
(programming)
However, it is very likely that affordable ‘off the shelf’ interfaces for web testing will become available very soon,
much as affordable ‘off the shelf’ interfaces for lab testing did in the 1980s
Ethics andweb-based testing
Ethics
In online tests, the ethical considerations are the same as those that apply to lab-based studies
For example, subjects must give informed consent, should be fully debriefed, can withdraw at any time and any
deception should be revealed and benign
Conclusions
Although not without its problems, web tests have a lot of advantages, particularly when used in conjunction with lab-
based studies
Recommended