Upload
daria-bogdanova
View
18
Download
0
Tags:
Embed Size (px)
Citation preview
13/12/2013
1
Research Seminar for
Educational Sciences
Prof. Dr. Chang Zhu
Department of Educational Sciences
Overview
• Quantitative research
• Qualitative research
• Questionnaires
• Interviews and observations
• Sampling
2
13/12/2013
2
• Make questions clear: word questions as clearly
as possible
• Keep questions short
• Collect demographic information based on the
need of the study
3
Questionnaire
Questionnaire
• Include only items that are relevant
• Define or explain them if some terms are
not familiar for the participants.
• Use examples if item format is unusual.
4
13/12/2013
3
Questionnaire
• Avoid leading questions: A leading
question is one that suggests a certain
response (either literally or by
implication) or contains a hidden
premise.
• Avoid ambiguous, and define or explain
them if some terms are not familiar for
the participants.
5
Questionnaire
• Cover letter or brief introductory or
directory words
� Why this questionnaire
� How the data will help the respondent and
the field in general
� How to respond
� Endorsement or approval by certain
admin/organization if applicable
6
13/12/2013
4
Questionnaire
I am happy to work in this school.
I am happy in my job as a teacher.
There is no better job than to be a teacher.
If I could choose again, then I would change the teacher
profession for another job. (R)
I like to be in my class.
I don't like to be in my class. (R)
I want to remain in the education for my job.
7
An example: well-being of teacher
Reliability
• E.g. measuring your weight 4 times a day
(is it reliable?)
• Testing your statistics knowledge 3 times a
week…
• Asking the well-being of a student with 5
different questions….
8
13/12/2013
5
Questionnaire
• Pilot testing
� A smaller sample
� Understanding of the questions
� Identify problems if any
� Provide suggestions for modification
9
Questionnaire
• Questionnaires (written)o Larger sample
�By mail
�Via Email
� online
• In-person question-and-answer (oral)o Smaller sample
� Telephone
� In person10
13/12/2013
6
Questionnaire
• Questionnaires
� Deadlines
� Follow up if possible (a reminder)
� Consider the convenience of respondents (e.g.
return envelope)
• Ethics
� Anonymity, confidentiality
� No harm, not threatening questions
11
Online survey
1
2
13/12/2013
7
Online survey
1
3
Online survey
1
4
13/12/2013
8
Online survey tools
• LimeSurvey
• https://www.vub.ac.be/survey/index.php?lang
=en
• Survey Monkey
• http://www.surveymonkey.com
• Qualtrics
• https://www.qualtrics.com/
• VUB/ULB Rekencentrum ([email protected])
• OSuCre1
5
1
6
13/12/2013
9
Interview
� Individual
� Group
� Focus group interview
– several individuals
– ensure all have their say
– individuals can interact, respond
– possibly lead to a shared understanding..17
Interview
• Understand themes of the world from the
subject’s own perspectives
• Capture the experiences and lived meanings of
the subjects’ everyday world (from their own
perspectives and in their own words)
18
13/12/2013
10
Interview
• Can obtain data that cannot acquire
from observation alone; gather in-depth
data about the experiences, views,
feelings of participants; explore reasons
• Can be structured, unstructured or semi-
structured
19
Interview questions
• Don’t ask two things in one question. E.g.
do you find online communication a
problem and the teacher should give more
guidance?
• Don’t ask Confusing or wordy questions
• Avoid unrelated questions
• Avoid Yes-No Answers.
20
13/12/2013
11
Interview questions
Interview questions
o E.g. ‘Can you tell me about…’
‘what do you think of…’
‘can you describe…’
‘can you say something more about…’
‘do you mean that….’
21
Interview
Interviewing
• Setting the stage, briefing
• Encourage the interviewee to describe their
point of views
• Follow up questions, clarifications
• If needed, summarize or round off (and ask
confirmation)
22
13/12/2013
12
�Interviewing
– listen more
– don’t interrupt
– avoid leading questions; ask open-ended
questions
– keep participants focused; follow up on
what they say
– do not judge or debate
23
Interview
� Collecting data from interviews
– take notes during the interview
– audio- or video-taping the interview
– write notes after the interview
– transcription of recording; write down
the data, subjects, and participant name
or codes
24
Interview
13/12/2013
13
• Some data can be observed in natural setting
• More objective information than self-reports
• Recording observations
• Field notes (description and understanding of the
research setting and participants, as detail as
possible)
• Recorded data and filed notes can be organized,
categorized and analyzed.
25
Observation
Observation
� Observational notes
• descriptive notes: detailed notes about
what occurred, what the observer sees,
hears, the actual setting, etc.
• reflective notes: about the observer’s
thoughts, impressions, personal reactions,
experiences
26
13/12/2013
14
Validity
� Validity in qualitative research
• descriptive: factual accuracy
• interpretative: are the meanings or words of
participants correctly interpreted?
• theoretical: how is the studied phenomenon
relate to a broader theory?
• evaluative: does the researcher report in an
objective, unbiased way?
27
Validity
� Strategies for ensuring the validity
– talk little, listen a lot
– record accurately
– begin writing early
– report fully
– write accurately
28
13/12/2013
15
A short discussion
• What kind of research method you would use for
your master thesis research?
• How are you going to collect your data?
• What would be your target sample?
• How many sample participants to reach?
2
9
Sampling
A. Defining a population
B. Selecting a sample
30
13/12/2013
16
Population
• What is your population of interest?
• To whom do you want to generalize your
results?
– All doctors
– All university students in Flanders
– All secondary school pupils
– Women aged 25-45 years
– Other
• Can you sample the entire population?
3
Sampling
• A sample is “a smaller (but hopefully representative) collection of units from a population used to determine truths about that population” (Field, 2005)
• Why sample?
– Resources (time, money) and workload
– Gives results with known accuracy that can be calculated mathematically
• The sampling frame is the list from which the potential respondents are drawn
– Registrar’s office
– Class rosters
– …..
3
13/12/2013
17
Sampling
• 3 factors that influence sample representativeness
• Sampling procedure
• Sample size
• Participation (response)
• When might you sample the entire population?• When your population is very small
• When you have extensive resources
• When you don’t expect a very high response
3
Sampling
34
13/12/2013
18
Sampling
3
5
TARGET POPULATION
STUDY POPULATION
SAMPLE
The sampling process…
POPULATION
SAMPLE
INFERENCE
13/12/2013
19
Sampling
Selecting a random sample
• Simple random sampling
• Systematic sampling
• Stratified sampling
• Cluster sampling
37
• Simple random sampling
38
13/12/2013
20
Sampling
• Systematic sampling
3
9
Sampling
4
0
• Stratified sampling
13/12/2013
21
Sampling
• Cluster sampling
41
Sampling
C. Selecting a non-random sample
• Convenience sampling
• Purposive sampling
• Quota sampling
D. Qualitative sampling
• Representative
• Often purposive
• Criterion sampling
42
13/12/2013
22
Sampling
Important to bear in mind:
• Is your sample representative of your target
population?
• Are the research results generalizable to your
target population?
• The degree to which the selected sample
represents the population is the degree to
which the research results are generalizable
to the population.43
Sampling
Sample size:
• In quantitative studies, samples should be as
large as possible; the larger the sample, the
more generalizable the results will be…
• In qualitative studies, samples usually
smaller, depending on the type of research
44
13/12/2013
23
Rules of thumb for determining
the sample size...
For smaller samples (N ‹ 100): Survey the entire
population.
The larger the population size, the smaller the
percentage of the population required to get a
representative sample
If the population size is around 1500: 20%
should be sampled.
If the population size is around 500: 50%
should be sampled.
Beyond a certain point (N = 5000): a sample
size of min. 400
Rules of thumb for determining
the sample size...
13/12/2013
24
Sample size
Sample size:
• No universally minimum sample sizes
• Some suggestions for quantitative studies:
if Population=<100, n=<100
if P=500, n~250
if P=1500, n~300
if P>5000, n~400-500 or more
47
Sample size
Sample size:
• No hard rules
• Some suggestions for Qualitative studies:
Can be 1- 60/70 participants
interviews: 15-30 or more, depending on population
size, should be representative of the sub-groups/
multiple contexts…
Case studies: 1-several, depending research needs
48
13/12/2013
25
Sample size
Sample size:
• Two general indicators to determine
whether a sample size is sufficient:
� Representativeness
� Redundancy of information (data
saturation) (in Qualitative research)
49
Sample size
Sample size:
• Avoid sampling error (occurs in the random
selection procedure)
• Be aware of sampling bias (eg. use of non-
random sampling..) (should be noted and
described in the research report)
50
13/12/2013
26
Task 1st semester
• Submit your research paper or the literature
review paper by 31 Jan. 5pm in Pointcarre
Assignment
5
1