43
CS2008 – Fundamentals of Research 1 © 2012 JIRO’S AWESOME NOTES CS2008 FUNDAMENTALS OF RESEARCH jeremiah wong Module Chapters 01 & 02 – Getting Started: Possibilities and Decisions 03 & 04 – Literature Review: Ethics 05 – Sampling 06 – Quantitative Research 07 – Qualitative Research 08 – Surveys 09 – Observations 10 – Experiments 11 – Content Analysis: Understanding Text & Image 12 – Writing Research & Review Module Objectives: You will be introduced to basic principles and methods of social research. Skills acquired upon completion of this course include being able to decipher and evaluate scholarly/social scientific research, to conduct basic empirical research and to improve analytical thinking. These skills are important for communication students. Even if you do not intend to be a researcher, you will still need to know enough to use other people’s research for decision- making and for your own work. In other words, research is essential to someone who wants to be a responsible professional. Like it or not, we are exposed to research findings all the time (e.g. poll results in the news). Without some background in research, we may not be able to judge the veracity of the reports. The course material also serves as essential basic foundation for those who pursue graduate study or careers in research. Course Assessment: 10% Lecture Panel, attendance and participation 10% Assignments 10% Quizzes 20% Term Paper o The paper will consist of identification of research problem, literature review, generation of research questions and a research design. A detailed guide will be distributed. 50%: Final Exam o The final exam will consist of multiple choice questions and a variety of other question formats requiring brief answers, in addition to an essay that will require advance preparation and background research.

JAN CS2008 Fundamentals of Research Ch1-14 Complete

Embed Size (px)

Citation preview

Page 1: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

1 © 2012 JIRO’S AWESOME NOTES

CS2008 FUNDAMENTALS OF

RESEARCH

jeremiah wong Module Chapters 01 & 02 – Getting Started: Possibilities and Decisions 03 & 04 – Literature Review: Ethics 05 – Sampling 06 – Quantitative Research 07 – Qualitative Research

08 – Surveys 09 – Observations 10 – Experiments 11 – Content Analysis: Understanding Text &

Image 12 – Writing Research & Review

Module Objectives: You will be introduced to basic principles and methods of social research. Skills acquired upon completion of this course include being able to decipher and evaluate scholarly/social scientific research, to conduct basic empirical research and to improve analytical thinking. These skills are important for communication students. Even if you do not intend to be a researcher, you will still need to know enough to use other people’s research for decision-making and for your own work. In other words, research is essential to someone who wants to be a responsible professional. Like it or not, we are exposed to research findings all the time (e.g. poll results in the news). Without some background in research, we may not be able to judge the veracity of the reports. The course material also serves as essential basic foundation for those who pursue graduate study or careers in research. Course Assessment:

• 10% Lecture Panel, attendance and participation • 10% Assignments • 10% Quizzes • 20% Term Paper

o The paper will consist of identification of research problem, literature review, generation of research questions and a research design. A detailed guide will be distributed.

• 50%: Final Exam o The final exam will consist of multiple choice questions and a variety of other

question formats requiring brief answers, in addition to an essay that will require advance preparation and background research.

Page 2: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

2 © 2012 JIRO’S AWESOME NOTES

01 & 02 – Getting Started: Possibilities and Decisions

• What is research? • What are the major fields of research in communication? • What are the major research decisions? • What are the major research approaches? • What are the starting points for doing research? • What kind of researcher are you?

1.1 What is Research? Research is a systematic process of -

• posing (asking) questions • answering questions • demonstrating that your answers are valid

1.2 Communication Research: Its Fields Human communication is complex. Difficult for researchers to examine a communication event by itself. So, researchers specialize on a part of it:-

• mass media • organizational or group • interpersonal • rhetoric and persuasion • communication technology

and choose a method to investigate (methodological or ideological) within their means à decision making. Major Decisions to make:- Approach

• Objective

• Social scientists assume a “real” world that can be agreed upon

• Common traits e.g. loyalty found in all people

• Used to predict human behaviour

Subjective • Phenomenologists

and ethnographers try to understand people’s subjective world

Page 3: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

3 © 2012 JIRO’S AWESOME NOTES

Researcher • How involved should I

get with my human ‘subjects’?

Dispassionate • Scientific trad. Is to

remain detached • Quest for

knowledge

Involved • Action Research • Specifically to improve

lives • Engaged contributor

to society Sample

• How many is enough to be accurate?

• Which size is suitable for my purpose / goal?

• Affected by resource constraints (time, attitude of informants)

Large • Gives numbers and

knowledge • Doesn’t mean

Understanding • Predict happening,

more people, closer to consensus

Small • Find out how people

feel • Goal is not about

generalization, but to get in-depth info

Data • “Majority Wins”, part

of a democracy. • In reality, it’s not an

either/or situation, but a blend of 2 ways to solve a problem

Quantitative • “Best” truth is the

most widely accepted

• Get numbers to find trend

• Yet, unable to capture more complex / subtle interpersonal things

Qualitative • Truth is listening to

stories that participants tell us

• Interviews • Get words,

descriptions, but no turning to numbers

Triangulation: using multiple methods providing multiple perspectives to ensure a good fix

Reporting • Different ways to

writing research • People have set of

values that they subscribe to

Objective • Scientific, e.g.

using APA style for writing.

• “Don’t matter how I think”

• Social scientists use hard stats to report and interpret

• Dispassionate researchers use neutral language

• E.g. “Subjects were recorded on video and their facial expressions subsequently…”

• Accuracy maximized by lack of subjective contamination

Subjective • Researchers use

qual. Language of their informants

• “Using a few participants to show how they feel”

• Involved researchers use “I” in writing

• E.g. “I lived with Thomas and his two children for 3 months and we formed a warm social bond…”

• Accuracy enhanced by including personal experiences and reactions

Both research and research writing are communication activities.

Page 4: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

4 © 2012 JIRO’S AWESOME NOTES

1.3 Posing, Solving and Peer Persuasion Problem Posing Problem Solving Peer Persuasion

• Have to decide what the research qsn is

• Hardest part

• How to best ans it • More than just

selecting a method and using it

• Can serve to amend inappropriate mtds and discovering new questions

• Research must be published to have value to the world

• Part of publication • Have to go thru peer

review by scholars before publication

2.1 Major Approaches to Communication Research Question: Are SCI students smarter than NTU students?

(Content Analysis) Functionalist

(Rhetoric) Interpretive

Critical / Radical

• Observe, measure from researcher’s perspective

• More objective • Doesn’t matter how I

feel

• Observe, interpret from participants’ perspectives

• More subjective

• Ask whose interests are advanced by communication

• Don’t rely on first hand data, don’t trust what they see

• Want to change society with their research

• Survey and rate SCI

Students • Not to be part of it • I have a trend in

mind, I devise an experiment to test if it’s true

• Observe daily life of SCI students

• Ask them how long they take to dress up / their activities

• Observe other sch’s students and compare

• Researchers’ report is subjective cos’ they report from the participants’ perspectives

• E.g. Does NTU culture stereotype minorities?

• Tell people “something is wrong”

Page 5: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

5 © 2012 JIRO’S AWESOME NOTES

Interests Objective Reality Subjective Reality

Regulation, stability, consensus.

FUNCTIONALIST

Rational behavior, social units based on consensus the ability to assess some kind of tangible social reality.

INTERPRETIVE

Communication must be understood from the individual (subjective) perspective.

Conflict, power, oppression.

RADICAL STRUCTURALIST

Basic conflict within society tangible differences between people.

RADICAL HUMANIST

Conflict and difference but - focus on the symbolic ways difference is maintained.

2.2 Starting Points for Research #1 What? – Interest Area

• Dependent on area of specialization o (mass, organizational, group, interpersonal communication vs. rhetoric /

argumentation) • Theories need to be supported by evidences à theory + observation • 3 thought processes link observation with theories:-

o Induction, Deduction & Abduction Scenario: Group of students drinking coffee and wrestling with their individual academic workloads. Induction (FORM) / “ITFT" Deduction (FROM) Abduction

Reasoning from Observation à form Theory to explain observation

From Theory à defining Observation to test theory

From an observed effect à reason out possible causes. [This results in HYPOTHESES]

“I see something happening, and I want to develop an explanation for it.”

“I have a theory about this, I want to test this out in real life.”

“I don’t know why it’s happening, but it could be because of (a hunch)…”

• Obs: Gender clustering – guys tend to sit with guys; vice versa

• Theory: Students have greater comfort among peers of same gender

• Theory: Women are more likely to discuss grades than men

• Obs: A test; record the no. of times the word ‘GPA’ appears in a convo for both sexes.

• Obs: My son keeps peeing, but what causes him to pee?

• Possible cause: Drinking too much water leads to peeing in bed (Hypothesis)

• Action: See which theory best explains the phenomenon

• Action: If frequency in women > men, my theory is supported.

• Action: Don’t drink anything 2 hours before bed…

Page 6: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

6 © 2012 JIRO’S AWESOME NOTES

• IF #1: True for all students?

• IF #2: True for all times?

• à Appropriate Sampling

• One further step: find a way to see which induced theories are correct

• Requires prior confidence that observations are enough and accurate

• Deduction is better than induction, because deduction leads to a specific observation to test hypothesis or statement about the relationship I expect to find.

Deduction

Induction

Rule All beans from this bag are

white All beans from this bag are white

Fact (Observation) Beans are from this bag These beans (oddly) are white

Case This bean must be white (Observation to test theory)

These beans are from this bag (Theory to explain Obs)

Abduction (Educated Guess, but not be an observation, might be a hunch) Deductive reasoning is NOT always functionalist, because in research, there can be a mixture of inductive & deductive. Interpretive research is usually inductive but not absolutely. #2 Why? – Starting with a Purpose Goals and Values of Research: Exploration • Curiosity-based research E.g. “I wonder why the

residents of 2 dorms have so different lifestyles?”

Description • Descriptions of what I am interested in (quantitative & qualitative)

• Qualitative makes compelling read, but still leaves people asking why

Quantitative: Statistical summary of students by age, gender, class, etc. Qualitative: What it means to be a student, to socialize with others.

Explanation • Attempt to answer “why” question to an observation

E.g. Obs: “Women more likely than than to socialize over coffee after class” [Female behaviour] à Exp: More women live on campus,

Page 7: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

7 © 2012 JIRO’S AWESOME NOTES

easiest way to organize grp proj over coffee. [Housing status]

Prediction

• Gives greater credibility to explanations

• Yet, doesn’t account for external factors (like bad internet at home)

Set an experiment for equal no. of off-campus students à see if more people at café are campus residents.

Control • To manipulate consumer behavior • E.g. to watch a show, to buy a

product, to open a mail

E.g. If you make your mail shape unusual, then people will open it…

Interpretation

• “In another person’s shoes” • From the POV of others • Reporting results of my research in

the language of my research participants

E.g. Meeting up for coffee à what does it really mean? (Getting hooked? Relaxing? Discussing work? etc)

Criticism • Understanding and explaining how comm is used to exercise and maintain power in groups, org and societies

In Org comm, does the org structure hinder or facilitate progress of certain groups within (males, females, 1st year students, minority races)

#3 How? – Starting with a Method The choice of research methods needs proper reasoning. Researchers conduct surveys, experiments, observation, interviews, focus groups and content analysis, depending on epistemology. Epistemology explains how communication should be understood. (How I get to know things) E.g. How do you know that your chair won’t give way when you sit on it? Tenacity “We’ve always done it that way.” I have always sat on it, and it

doesn’t give way. Intuition “It just seems right.” It seems sturdy. Authority “Because X said so.” Because the factory’s CEO said it

can withstand 100kg of weight. Empiricism “Because I’ve seen it.”

(observation only) Because I see that my friend didn’t fall when sitting on it.

Rationalism “It is logical.” A chair is supposed to hold the weight of anyone sitting on it.

Scientific method

observation and testing. This chair has been tested by the authorities to be stable and strong.

Serendipity the accidental breakthrough A fat guy fell on the chair, but it was still intact.

Page 8: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

8 © 2012 JIRO’S AWESOME NOTES

#4 Starting with a World View

Worldview I Worldview II • Human behavior is predictable,

objectively measurable and can be generalized

• Assumes that

• Human behavior is individualistic, unpredictable and subjective

• Assumes knowledge is constructed by interactions between people

• Researchers make generalization that holds through space and time

• Researchers attempts to describe and assess the subjectivity and individuality of human comm, instead of trying to discover universal norms

• Nomothetic approach • Idiographic approach • E.g. Advertising / Audience research,

to find rules that predict advertising, marketing successes. (What kind of TV commercial styles attracts sales from TV audiences)

• E.g. Find out how consumers respond subjectively to media content (To understand why some TV viewers develop a close relationship to comedians and not others)

Research methods must logically follow either worldviews (human assumptions).

Worldview I Worldview II • “What type of survey can I run?” • Quantitative method

• “What focus groups or interviews will I need?”

• Qualitative method Ontological questions: addresses nature of human comm and what is it we really observe when we observe it. E.g. To what extent do we make real choices? Is it really a choice to go to school? (We can’t really ‘choose’, we mostly go cos we have to). 2.3 Operationalizing Constructs In Worldview I, we need to identify key constructs (ideas and concepts) and operationalize them (define them in a measurable way).

Page 9: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

9 © 2012 JIRO’S AWESOME NOTES

Constructs (cannot measure) à Operationalize à Variables (Can measure) #6 Starting from the Work of Others: Literature Review Do regard the work of others, so as to:-

• Avoid doing redundant research (stuff that has aldready been done) • Avoid missing out on relevant research methods and helping findings • Avoid missing out on what is ‘good research’

#7 Hypotheses Hypotheses: Making Predictions Hypothesis (H): statement about a relationship between variables Two-tailed hypotheses (Non-directional)

Relationship between 2 variables, but no direction specified

H1: There is a relationship between involvement in video games and academic performance.

One-tailed hypotheses (Directional)

Relationship + Direction (Already Predicted)

H2: As time spent in playing video games, academic performance decreases.

Null hypotheses (H0) No relationship between variables

H0: There is no relationship between involvement in video gaming and academic performance.

#8 Research Questions When preliminary research is insufficient to give certainty for predictions, use research questions (RQ). Open-ended RQ Asks simply if there is a

relationship between variables

RQ1: Is there a relationship between involvement in video games and academic performance?

Closed-ended RQ Focuses on direction of relationship

RQ2: Does academic performance decline as involvement in video gaming increases?

RQs good for exploratory studies in absence of evidences. Open-ended RQ Close-ended RQ Relationship (Hypothesis) + evidence

+ more evidence

Page 10: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

10 © 2012 JIRO’S AWESOME NOTES

#9 Starting Without a Question Because a specificity of a question or hypothesis could blind us to relevant phenomena à hinder success of research. Groping through data collection until we make sense of what we collect à making no assumptions until an assumption can be made. Methods and findings will change as study progresses. 2.4 Summary & Key Terms Researchers differ in their:-

• basic assumptions about communication (ontology and World View), • views on how communication should be understood (epistemology), • views on how theory and research relate, • reasons for doing research,

and therefore, their research methods. Research – a systematic process of asking and answering questions about human communication. Researchers specialize by:-

• interest area • research method(s).

Research methods reflect:- • Researchers’ interest area • Researchers’ assumptions about the

nature of human communication.

Research begins with – • “What?” “Why?” “How?” • Epistemological and ontological assumptions. • A research purpose. • World View or basic assumptions. • Links between observations and theory. • The work of others. • Research questions and hypotheses.

Epistemological assumption explains how

communication should be understood (How I get to know things)

Tenacity, Authority, Empiricism, Rationalism, etc

Ontological assumption addresses nature of human comm and what is it we really observe when we observe it

What really is it? “What is an attitude?”, “Do we really have real choices?”

World View WV1 (Objective) or WVII (Subjective)

Page 11: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

11 © 2012 JIRO’S AWESOME NOTES

03 – Literature Review: Ethics 3.1 Ethical Considerations Ethics: study of right and wrong. Appropriate behavior towards research participants. CR ethics: to protect well-being of human participants Many research designs / methods are legal, but not necessarily ethical. E.g. To let audiences ‘fake’ their responses to a speaker to study his reaction. E.g. To exposes audience to sexually explicit content to study their response. E.g. To spread a false rumor to test the pervasiveness and speed of information spread in an organization. 3.2 Issues with Communication Research Honesty • deception is part of legitimate and professional research studies

• researchers have ethical responsibilities to readers and participants

• Finding ‘nothing’ or counterintuitive results is still contributive and can be reported

Confidentiality & Anonymity

• Conf: Not releasing any info that identifies my participants

• Ano: Data collector will NOT know who participated

• Violation is a ethical and legal issue

Using “Respondent A” and “Respondent B” to refer to people

Generalisation • All recruited participants are volunteered into my research • Responses are naturally ‘biased’ towards me • So I cannot say the results talk about the ‘non-volunteer’ group too

Debriefing • Deception must be revealed to participants as follow-up (Ethical obligation)

Literature Review

• Summary & evaluation of other researchers’ work on my topic • Show how my rsc contributes to knowledge • Reviewing summary may lead to oversimplification • Publishing proprietary info is a ethical and legal violation

Acknowledging Others

• Who are the ‘others’ that ‘contributed’ to my research

Appropriate Language

• Documents be understandable to layman (High School level) • Writing can actually misinform (ethical)

Plagiarism • Illegal and unethnical

• ‘Used Goods’ à other researchers read my plagarised data as if it were new

Page 12: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

12 © 2012 JIRO’S AWESOME NOTES

3.3 Ethics Classic Ethic Positions Contemporary (Formal) Codes of Ethics

• Judeo-Christian: Do to others what you wan others to do unto you

• Kant: Behavior is valid if applicable unto universal rule

• Utilitarianism: Small ‘sacrifice’ justified for ‘greater good’

• Veil of Ignorance: Blinding ourselves to the roles (researcher or participant) in the research

• Professional standards of behavior for researchers

• E.g. Nuremberg Code, Declaration of Helsinki, Belmont Report

3.4 Peer Review for Ethical and Human Participation Qualified researchers assessing each others’ work à provide check on ethics of research Informal (over coffee) or formal (Institutional Review Board)s IRB is a panel to review research proposals based on their impact (physical & psychological) on humans. 3.5 Researchers’ Involvement with Participants in CR Low High Dispassionate Observer People à Subjects People à Informants Involved in Participants’ condition

People à Participants People à Collaborators

Page 13: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

13 © 2012 JIRO’S AWESOME NOTES

04 – Reading Research 4.1 Library Research Purpose of any research à ADD to knowledge Need to know what has been done before, so that:-

1. will not duplicate research 2. will add to knowledge 3. will give me a broader view of my field 4. suggest proper research methods 5. have confidence of own contribution to field

Look at literature – published scholarly research done to academic standards. Method Focus group

Ethnography Field interview Experiment

Ethics How to deal with minors, manage issues of deception etc. Language & Style Learn of the way to write for a particular field Inspiration ‘Breakthrough’ paper that makes a difficult concept clear 4.2 Finding Relevance & Quality in Literature Search

Relevance Quality • Relevant Info – info that is

immediately useful • Quality info – Info that is credible,

reliable and meets academic standards of research community

• Develop good vocab, know the difference that different search terms can bring

• Obtained thru peer reviewing and refereeing (reviewed or refereed by other researchers in same field)

Info can be relevant but not of academic quality, or vice-versa. So my research must have BOTH. Must focus on communication. V Relevant X Not of Quality

X Irrelvant V Of Quality

E.g. National Enquirer’s report about communicating with aliens à not of scholarly merit

E.g. New England Journal of Medicine’s research à not relevant to corporate rhetoric

Page 14: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

14 © 2012 JIRO’S AWESOME NOTES

4.3 Databases vs. Search Engines Databases =/= search engines.

Databases Search Engines • To find academic journals • Have defined no. of entries • Contain peer reviewed articles • No irrelevant ‘hits’ • Results have high cred. •

• (+) Ease of use, Easy access • (-) Unmanageble amounts of data • (-) Questionability of results •

• Need to know search terms; wrong terms à wrong results

• More sophisticaded interface • Can focus in on specific databases

(Business, Mass Media etc)

è Choose databases over search engines 4.4 Scholarly, Trade & Popular Publications Peer review to check professionalism, contribution and violations. Scholarly Trade Popular Format Headings of ‘Abstract’,

‘Method’, etc In between Published w/o refereeing

process; no formal heading References Have references No references Frequency Published monthly,

quarterly Published daily / weekly

Examples “Journal of…” Newspapers / mags Sources Primary (Original; have

citations) Secondary (Only have summaries)

4.5 Gathering Information

Must Record Should Record • Author’s full name • Title of book / article • Date of publication • Edition number • Page number • URLs for websites & date of info

download

• Method – how rsc was conducted • Results – what were the results and

conclusions • Participants – who or what was

studied • Unique aspects of the study

Page 15: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

15 © 2012 JIRO’S AWESOME NOTES

4.6 Writing the Literature Review Purpose: show my justification for doing that research, and how my research contributes to existing knowledge Use either chronological or pro-con structure. 4.7 Mixed Methods (Readings) Combines quanti and quali research methods.

1. What types of questions should be presented? 2. When & what information is most needed to convey the nature of the study?

You can write a mixed methods study in 3 ways:

1. Write separate quanti and quali 2. Write separate quanti and quali + mixed method qsn 3. Write only a mixed method qsn

Quantitative Qualitative Mixed Method

It was hypothesized that there would be no significant difference between students in the middle school and those in the junior high in attitude toward science as a school subject.

What differences currently exist between the middle school instructional strategy and the junior high instructional strategy at this school in transition?

How do the interviews with teachers, the principal, and university consultants help to explain any quantitative differences in achievement for middle- school and junior high students?

Page 16: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

16 © 2012 JIRO’S AWESOME NOTES

RQs and Hs narrow the purpose statement (specific topic and goals of a research) and become major signposts for readers.

Qualitative researchers Quantitative researchers • Ask > one central question and

several subquestions • begin questions with words such as

how or what and use exploratory verbs, such as explore or describe

• pose broad, general questions to allow the participants to explain their ideas

• focus initially on one central phenomenon of interest. The questions may also mention the participants and the site for the research.

• RQs à inquiry about relationship among variables

• Does _________ (theory) explain the relationship between _________ (independent variable) and _________ (dependent var), controlling for the effects of _________ (independent var)?

• Hs à predictions of expected rsn

among variables • There is no significant difference

between _________ (independent var) on _________ (dependent var).

Central Qsn: broad question that asks for an exploration of the central phenomenon or concept in a study.

• Write either RQ or H • Include variables categorized into

groups for comparison; independent and dependent variables are measured separately

• Proposals à RQs • Formal statement of research à H • H are predictions about outcomes of

results:- o Alt H: specifying the exact

results to be expected (more or less, higher or lower of something)

o H0 (null form): indicating no expected difference or no relationship between groups on a dependent variable

• Researcher writes independent variable(s) first à followed by dependent variable(s)

• Begin with descriptive questions à followed by the inferential questions that relate variables or compare groups.

 

Descriptive Qsn Inferential Qsn How do the students rate on critical thinking skills?

Does critical thinking ability relate to student achievement?

A descriptive question focused on the independent variable

An inferential question relating the independent and the dependent variables

 

Page 17: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

17 © 2012 JIRO’S AWESOME NOTES

 05 – Sampling    Sampling: process of selecting individual units for study. Not practical to collect data from every individual in a population. Sampling can be either nonprobability or probability sampling. Sample size determined by homogeneity of population and level of confidence. 5.1 Sampling Decisions Sampling lets us make intelligent estimates from a large / unknown quantity.

Population Census Sample Every indv of a type I want to study

A study of every member of a population

Selected segment of a population presumed to represent the population

5.2 Nonprobability Sampling Decision based on professional judgment. Cannot claim generalizable ability. Assumption that probabiity Advantages Disadvantages Convenience Sampling

Good for pretesting Due to constraints of time or money Convenience Sampling

sample not representative

Purposeful / Judgmental Sampling

Belief that a specific person or media content will meet the criteria of the researcher

Quota Sampling

Replicating in the sample what is important in the pop.

Not randomly sampled. Chosen by researcher’s judgment, so may be biased.

Network or ‘Snowball’ Sampling

Relying on members of a network to introduce other members. Complete sample size of populations.

Dependent on willingness and ability of others to identify people to me.

Volunteer Sampling

Will provide information and volunteer enthusiasm

Findings will be biased since I have no idea what non-volunteers will say.

Page 18: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

18 © 2012 JIRO’S AWESOME NOTES

5.3 Probability Sampling Probability sampling reduces bias of convenience à make generalisations of a population from a sample Random Selection Process: Using a mechanism (over which the researcher has no control) à every unit has an equal chance of being selected. Sampling Frames Sampling Units Master list from which a probability sample is selected

Units selected for study. Usually individuals, or couples, corporations etc.

Advantages Disadvantages Random Sampling (Simple Random)

Reflective of actual demographic, not necessarily diverse

May or may not reflect the population that they are drawn from Randomness à possible to entirely leave out a subgroup

Stratified Random Sampling

To ‘force’ all groups to be counted in my sample Proportional representation

Systematic Sampling

Selecting a randomly decided nth unit, and every nth there after. N = sampling interval.

Findings may be affected If pattern exists in original population which matches the sampling interval

Multistage Cluster Sampling e.g. randomly select housing units à randomly select floors à randomly select units

Sampling larger units first, then the smaller units thereafter. Relative ease of identifying people (using states instead of millions of people).

Potential bias at every stage, cos no two stage types (e.g. states, towns, city blocks) are equal à result in over/under representation.

5.4 How Big a Sample do I need? Larger sample size reduced sampling error, but the extent of reduction depends on the homogeneity of population. Homogeneity of Population Level of Statistical Error I can tolerate Refers to range of diversity. E.g. 100% voting for WP à Totally homogeneity 75% voting for WP à Less homogeneity 50% voting for WP à Minimal homogeneity 25% voting for WP à More homogeneity (since majority are not voting for WP) The less homogeneity à need a larger sample to ensure diversity is captured.

Larger sample size à more likely to reflect the population (less sampling error) But dependent on

Page 19: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

19 © 2012 JIRO’S AWESOME NOTES

5.5 Method and Method Issues in Sampling Method selected affects ability to form an appropriate sampling frame. Method Issue Postal Sampling • Excludes the homeless and transient

• Cannot reach people who have moved Phone Sampling (from phone directories)

- Convenient - Easy

à Random Digit Dialing

• Many phone numbers are unlisted • Will exclude those owners of unlisted numbers • Excludes poor households that cannot afford telephone

lines

• Many sequences of numbers not in use • Excludes people who abandon traditional telephony for

internet telephony (e.g. Skype) Internet Sampling • Excludes people who don’t know how to use the internet

• No comprehensive directory or format à unequal chance of selection

Special Population Sampling à Membership Lists à Subscription Lists

• Tricky when accessing special populations (e.g. military families, prisoners, mentally ill people)

• Difficult for organisations to disclose list of members • List will not be exhaustive (not all subscribers to a

magazine will be desirable units, not all desirable units are subscribers)

Page 20: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

20 © 2012 JIRO’S AWESOME NOTES

06 – Quantitative Research 6.1 Why numbers? Assigning numbers to things lend precision to an imprecise world; basis of all statistical analysis. Validity: Need to measure what they’re supposed to measure Reliability: Measure consistently

Numerals Numbers • Meaningless labels • e.g. 13, 2010, 64 could be street

addresses, a brand name, plane number, etc

• Have value and are relative to phenomena

• e.g. 1 to 5 indicate increasing level of agreement in a survey

Advantage of numbers:- Able to discriminate accurately

• To see if there are real differences between groups

Able to generalise • Allows us to generalize • Large figures gives us more confidence

Easy to process • Easy to input responses as number choices than

essay replies 6.2 Introduction to Scales Measurement: process of finding out if our focus of interest has more or less of an attribute we are interested in.

• E.g. whether people watch 5 or 50 hours of TV in 1 week Some cases are harder to measure than others

• E.g. Whether ‘rich’ or ‘poor’

Page 21: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

21 © 2012 JIRO’S AWESOME NOTES

6.2 Research NOIR N Nominal

Labels • Labeling / classification

• Coding (subbing words for computer numerals 0, 1) is still nominal measures

• NOT numbers à Don’t measure anything

• Can only generate percentages

e.g. of nominal measures “Buddhist, Christian, Muslim, Hindu”, “413, 455, 560, 934” “Male – 1, Female - 0” “30% male, 70% female”

O Ordinal Ordered Labels

• Indicate level of progression • One category has “more” than

the previous • Still vague and imprecise à

hard to quantify how much ‘more’

e.g. of ordinal measures “Freshmen, sophomore, junior, senior”, “first, second, third”

I Interval Meaningful Distances

• Assuming equal intervals between points on a scale

• Common in quantitative rsh

E.g. Likert scale, Semantic scale

R Ratio Absolute Zero

• Have a “true” zero, referring to absence of that attribute

• Top of NOIR hierarchy, since they allow us to use most sophisticated statistical tools

• Can also be percentages

E.g. Class attendance, going to movies can be ‘zero’, as in never happened Copy from Photo.

Example: how age can be measured at 4 levels of NOIR.

Nominal Ordinal Interval Ratio Parent, Child Child, adolescent,

adult __ 0-4 __ 5-9 __ 10-14 ___15-19

Age in years _____

Can classify and rank Use as reporting and decision making tool è NOIR is relevant because some attributes can only belong to one NOIR category. E.g. Sex is nominal, only 1 (male) OR 0 (female), no 0.74 (??). à can only be classified, not computed.

Page 22: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

22 © 2012 JIRO’S AWESOME NOTES

6.3 More than NOIR à Reliability and Validity Reliability + Validity = Credibility of measuring instrument Reliability: to repeat measurements and comparing extend of results (same or not) Testing Reliability:-

Test-Retest Intercoder / Observer Reliability • “Do again and again” • Test administered to a group and

repeated 1 or 2 weeks later • Test scores à Correlation scores à

Reliability Coefficient

• We need observers to agree they are observing the SAME thing & to RECORD the same thing

• We have to first operationalize categories

Ranges from 0 and 1.0 (perfect E.g. Define ‘unarmed’ aggression so all observers are on the same page. Cos different people have differing definitions of what’s considered ‘armed’ and ‘aggression’.

Reliability without Validity is useless. Testing Validity (am I measuring what I want to measuring):-

Content Validity Looks OK

Construct Validity Theoretically OK

Criterion validity Tests OK

Face Validity: as seen by the lay person à Need to find the ‘right’ people to ask questions

Convergent Validity: a close rsn to similar measures à e.g. identification, loyalty, commitment à Test on coolness of Sg should correspond to test on happiness of Sgpreans

Concurrent Validity: score on one measure to correlate highly with other measures à e.g. anger on test A will correlate with anger in test B

Expert / Panel Validity: as seen by more knowledgeable insiders like comm scholars à Preferred since its passed peer approval.

Divergent Validity: should not show relationship to dissimilar measures à e.g. high on loyalty, low on individuality

Predictive Validity: predict ‘real world’ outcomes à e.g. High GPA predicts success in high school

Page 23: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

23 © 2012 JIRO’S AWESOME NOTES

6.4 Two Common Measurement Scales Allows us to assign numbers to respondents’ answers & make summary statements about their responses.

Lickert Scale Semantic Scale • Framed as a statement • A different statement for each scale • Vary between 5 to 7 points • Standard responses (Strongly

disagree, Disagree, Neutral, Agree, Strongly Agree) à each ans given a numerical value

• Pairs polar opposites (e.g. Powerful, Weak; Expensive, Cheap)

• Same questions for many scales • Multiple scales per concept

• Respondants check the best ans à answer recorded as a score

• Respondents to decide between the two concepts, where their opinion lies

• 1. Hybrid Cars for powerful.

SA A N D AD __ __ __ __ __

1. Hybrid Cars Powerful __ __ __ ___ __ Weak Expensive __ ___ ___ __ __ Cheap

• More difficult to do • Must pretest words (to see if they are

really the opposite of each other)

6.4 Drawbacks / Limitations of Quantitative Research Both scales make assumption of ‘equal distance’ between points à may not be true psychologically The test becomes the thing. E.g. IQ tests become a test of IQ. While in reality IQ is much more than answer human-worded quizzes.

Page 24: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

24 © 2012 JIRO’S AWESOME NOTES

07 – Qualitative Research Communication researchers use both quantitative and qualitative methods because human interaction is more complex and intricate than can quantified with measuring devices. Qualitative research methods are sensitive to social constructs, and explore social phenomena thru emphasis on empirical, inductive and interpretive approaches. Studies the communication environment of recipients, allowing us to explore social phenomena. 7.1 What is Quantitative Research? Best to understand it in relation to quantitative research.

Quantitative Qualitative

Changes form and content of human interaction

Preserves form and content of human interaction

Subject to mathematical transformation Not subjected to mathematical transformation Converts, separates and isolates into numerical form

Does not become numerical data

Includes participant observation, interviews, focus groups, narrative analysis and ethnography. Discourse (naturally occurring talk or gesture) is captured in the form it is originally in (no change).

Subjectivity Inter-subjectivity à the interpretive research processes used to make the subject of interpretation meaningful à Opposed to common definition of being individualized.

à the social accomplishment of how people co-construct and co-experience the interaction of social life and their rules for it).

All Qua. Research methods:-

• Interest: All have theoretical interest in human communication process • Context: All treat study of comm. as socially situated human actions and artifacts • Subjects: All use human investigators as primary research instrument • Medium: All rely on textual, written forms for coding data

è Qua. Reports are very different from quanti. Reports

Page 25: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

25 © 2012 JIRO’S AWESOME NOTES

Unlike quanti. reports that talk about causality (what causes what), Qualitative reports suggests mutual simultaneous shaping (everything influences everything else) à impossible to identify specific causality Interaction is non-directional à cannot say what causes what à quantitative way of dissecting events into parts is inappropriate. All qua. Research methods:-

• Treat human interaction as data • Have researchers ‘there’ in person to observe or obtain detailed records of interaction • Participants are natural actors in their settings

è Hallmark of qua. Research methods 7.2 Inductive Analysis Qua. Methods rely on inductive reasoning (specific à general).

1. Researcher become intimately familiar with field of interaction 2. Needs time to think and assimilate into the surroundings 3. Take (better) description of surroundings 4. Analysis and interpretation 5. Synthesis and making whole 6. Writing the research document

Qualitative research has a process with a beginning, middle and end, but it is more cyclical then linear à researchers need to return to research literature and refine their design and topic (iterative). 7.3 Credibility in Qualitative Research Reliability and validity is difficult to apply to qua. research à use credibility instead. Credibility: extent to which interpretations can be validated as true, correct and dependable. Credibility is important because qualitative research can produce many interpretations.

Page 26: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

26 © 2012 JIRO’S AWESOME NOTES

Researchers use triangulation (use of various methods to bring credibility to their findings): Data triangulation • Using various sources (e.g. survey data, interviews,

focus groups) • Using several data sources

Investigator triangulation • Using various researchers / evaluators à greater confidence in findings since results are not dependent on a single person

Interdisciplinary triangulation

• Reseachers from various disciplines work together • They can question each other’s bias and perspectives

7.4 Conceptualizing RQs for Qualitative Research The process is inductive since theory is developed during data analysis. Quantitative processes already have a theory in place, and then test it out. Qua. RQs are broadly stated and nondirectional. e.g. RQ1: How do nurses managers define their roles? Qua. RQs:-

1. provide us with a focus 2. Allows us wide latitude to follow paths which only appear as we collect data

è Qualitative studies are structured around an overall RQ. Qua. RQs:-

• start with ‘how’ or ‘what’ à explain, seek, explore, describe a process • are worded nondirectionally (e.g. X affect, X influence, V play a role) • Reference the research site (context)

Page 27: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

27 © 2012 JIRO’S AWESOME NOTES

7.5 What counts as Data? Difficult to say what counts as data since anything captured can be data. Researches use two issues to help them decide what is data:- Interpreting Meaning Level of Evidence (data) 1. Researcher Construction Interpretation based on researchers’ subjective position à evidence is fully the construction of the researcher 2. Subjective Valuing Mix of objective (tangible artifacts) and subjective elements (with some researcher interpretation). 3. Contingent Accuracy Relies on tangible artifacts believed to be accurate representations of the phenomena. Best selection requires least interpretation.

• Can be one-word or lengthy desc.

• Microlevel: can stand on its own • Macrolevel: broad-scale and have

many similar data belonging to the same classification

• Midlevel: somewhere between the two

E.g. Micro >> Mid >> Macro Direct quotes >> Interaction process >> Org norms A book >> TV genres >> Cultural values à Some qua. studies focus on the same level of evidences.

7.6 Making the Case for Qualitative Research

Advantages Disadvantages Able to ‘catch’ things that the participants themselves are oblivious to e.g. 旁觀者清

Only can work for accessible communicative environments e.g. Parents talking to children about sex are not accessible env

Able to provide info about those who cannot or will not speak for themselves e.g. Babies, elderly cannot be ‘tested’ in lab

Inaccuracy since people who are observed become conscious à change their behavior

Can supplement info provided by quantitative analysis

Observation filtered thru interpretive lenses of researchers

More time consuming than quanti à researchers may choose the convenient way to research à not get the whole picture

Page 28: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

28 © 2012 JIRO’S AWESOME NOTES

Threats to credibility:- Inaccurate or incomplete data

• Inability to make audio or video recording

Problem of our interpretation

• (Skewed by our own perception or knowledge), or interpretation of others (different people see and think differently)

• à Solution: listen to participants’ POV, see if can justify the differences btw ppl

Threat to Theoretical Validity

• Data that do not support my finding

• à Solution: need to analyze them, not dismiss them Selection Bias / Reactivity Bias

• Selection Bias: when certain data ‘stands out’ to me • Reactivity Bias: when participants’ interaction becomes affected

by presence of (e.g. after knowing that I’m the researcher, will they ‘react’ differently?)

Solutions: Use triangulation, feedback; check each others’ behavior etc. 7.7 Key Similarities and Differences in Qua & Quanti

Similarities Differences (Qua vs Quanti) • Both rely on empirical evidence

external to the researcher (not from himself, but something he can ‘point’ to, to make a conclusion).

Observation • Qua: Text • Quanti: Numbers

• Both provide useful info for describing, understanding and explaining human behavior.

Selection of participants to be studied • Qua: Purposely-selected • Quanti: Can use probability selection

Contextuality

• Qua: Deeply situated in environment • Quanti: lab conditions remove the

environment Location of Argument

• Qua: replies on interpretation of researcher

• Quant: relies on formal logic based on statistical analysis

Interaction Reality • Qua: Assumes many realities that

cannot be quantified • Quanti: Assumes a reality that can

be easily measured and tested

Page 29: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

29 © 2012 JIRO’S AWESOME NOTES

08 – Surveys Survey: series of formatted questions delivered to a defined sample of people, expecting their responses immediately or very soon. Questionnaires: specific set of questions that respondents answer. 8.1 Advantages and Disadvantages of Surveys

Advantages Disadvantages • Can answer large no. of questions

rapidly • Many people can be surveyed rapidly

thru use of technology • Can make generalizations with a

known level of understanding

• Gives no understanding behind numbers

• Most surveys do not allow assessing of causality

• Increasing unwillingness of consumers to participate in surveys

• Responses may not be entirely truthful

8.2 Types of Surveys Cross-sectional • Capture responses at ONE

point in time • Slice of life

à Results may differ another time à Soln: Decide that people’s attitudes are ‘unlikely’ to change, or use longitudinal studies

Trend • Measure same items over time but draw different samples from population each time

• People are replaced should some drop out of study

à New people may differ in opinion.

Panel • Same group of individuals sampled over time

• No changes in people composition over time

à High attrition rate, group gets smaller over time

Cohort • Groups of people defined by a common event (e.g. grad class of 2011)

Cross-lagged • Measure a dependent and independent variable at 2 points in time

• Tells us about causality

Page 30: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

30 © 2012 JIRO’S AWESOME NOTES

8.3 Question Formats Open-ended

• Can get insights that cannot be gotten with highly structured questions

à Time consuming to code and analyze

E.g. Where do you live? ____ E.g. How do you feel? ___

Dichotomous • Force people to choose 1 of 2 possible answers

à Life choices are rarely so straightforward à Only suitable for clear-cut choices

E.g. What is your gender? __ Male __ Female

Additional Category (+)

• Provide several possible answers and, can select multiple and rank

• ‘Select as many’ à to look for patterns of response

E.g. I am willing to:- __ Go filming __ Pick up trash __ Fly a kite __ Make a cake

Intermediate Category

• Likert Scale o Always statements, never

questions o Similar statements to find

out new things • Semantic Differential Scale

o Topic, Concept, then scales with words of opposite meaning

E.g. Strongly agree, agree, neutral, disagree, Strongly disagree E.g. Good _ _ _ _ _ Bad

8.4 Common Problems with Survey Wordings Leading Questions

• Force people into an assumption that may be false

• Lead people to a particular answer

E.g. Why do you think the school administration is unethical? E.g. When did you start to plagiarizing your research paper?

Double-Barreled Questions

• Asks two questions but allow for only one answer

• Need to split them into 2 separate questions à format as yes/no or Likert qsns

E.g. Do you think NTU is an asset or should it close down?

Negative Wording

• Don't use statements in their negative states, e.g. ‘not’

• Easy to miss that word out

E.g. Chinese should not be included in the University curriculum

Double Negative

• Negative wording + Double barreled question

E.g. Does it seem impossible or not that pigs cannot fly?

8.5 Guiding respondents through Surveys Funnel format: Broad à Specific Inverted Funnel format: Specific à Broad

Page 31: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

31 © 2012 JIRO’S AWESOME NOTES

8.6 Filtering (rerouting) Questions and Instructions Some questions are irrelevant to certain people. E.g. Questions of Facebook related questions do not pertain to non-users of internet.à Route the respondents pass these questions to the next relevant ones. If this home has an Internet connection, please continue with question 6. If it has no Internet connection, please go to question 18. 8.7 Pros and Cons of Survey Methods Phone Most households have phones

Can survey large samples quickly Shallow; Can only ask few questions People don’t pick up

Mail Give people time to think and answer in any order

Low response rate Cannot know who did the survey Only captures literate people

Internet Quick and cheap Interactive; can have video and audio Can get sensitive information

Answers may not be truthful Cannot generalize to public Need to drive traffic to the survey Cannot know who did the survey

8.8 Pretesting If it can be misunderstood it will be.

• Aesthetics and design of questionnaire • Logic and flow of wording. • Words – specific meaning(s)

Construct Validity: test using convergent (e.g. ‘Good’ must tally with ‘Kind’) and discriminant validity (e.g. ‘Good’ must be inverse to ‘Bad’). 8.9 Improving Response Rates

• Advance letter or phone call • Reminder letter or phone call • Reply paid envelopes for mail • Small gift or donation to charity • Follow up ‘thank you’

Page 32: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

32 © 2012 JIRO’S AWESOME NOTES

9.1 Survey / Experiments vs. Observations

Survey / Experiments Observations • Require a RQ to begin • Adds indv to increase sample size • Respondents are similar • Search for theories

• Does not require RQ to begin • Adds indv to gather new views, diff

data • Respondents are different and

subjective • Search for meaning

à More valid, but less reliable. Reliability: the ability to repeat a measurement to study a trend. E.g. Observation as a research method is less reliable than experiments because human behavior (even in the same individual) can also change over time.

Validity: the ability to measure what I want to measure. E.g. Observation as a research method holds more validity participants are in their real-life settings. People behave more ‘naturally’, so the data is hence more accurate. Yet, since participants are not randomly selected, people chosen cannot represent the population (if that is what I want to measure).

Ethnography: study of human behavior (observe, describe and interpret) How + What Level of involvement? 9.2 Researcher-Participant Relationships The level of engagement between researchers and participants. Researcher-Informant relationships as:- Complete Observer Observing as a

participant Participating as an

observer Complete

Participant - No interaction with informants

- one-visit scenario - low involvement

- Some time is spent - Trust develops btw rshr and participants

- Very close participation - Rshr’s role is unknown to them - Rshr pretends

à Not enough understanding of situation à If friendship develops, one might be afraid to ask hard questions

à Become unable to function well as observers à Ethical risk?

Page 33: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

33 © 2012 JIRO’S AWESOME NOTES

9.3 Interview Considerations Interviews: series of questions to get information that rshrs are interested in. Questions can be specific or general. Effective interviews need to have the following considerations: Setting • Rshr’s office or participants’

natural setting? Clothes worn?

à Rshr’s office is easier to manage, but may have less honest answers.

Sensitivities • Cultural, religious, tech sensitivities

à People scared of camera, women afraid to talk to men etc.

Structure • Full structured • Semistructured

• Unstructured

à Rshr merely records answers à Rshr asks some questions to allow further elaboration à Rshr starts with generic questions

Sequence • Order of questions • Funnel or Inverted Funnel

Funnel: Broad to Specific Inverted Funnel: Specific to Broad

Question Types

• Descriptive: Overview of everyday occurrences • Structural: Relationships among terms that informants use • Contrast / Ranking: Explain differences / similarity / importance of

informant’s concepts Why use interviews? Interviews are personal. 9.4 Focus Groups 6-12 people led by moderator with good interview and discussion skills. Need to select a diverse set of members. 9.5 Saying =/= Behaving People can say one thing and do another. But we can check people’s behavior and relate to what they say thru unobtrusive measures. E.g. If all respondent say they never text when driving à Check the traffic on the road à Provides a general sense of whether the self-reports of respondents are credible.

Page 34: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

34 © 2012 JIRO’S AWESOME NOTES

9.6 Categorising Qualitative Data as Analysis Categorisation: Identifying each piece of data as belong to certain categories determined by rshr or by data set. How to categorise?

Fixed Coding Flexible Coding Assign items to specific, preassigned categories

Start with theoretically assigned categories that may change as data come in

No pre-fixed categories, allow new categories and theories to emerge

à No room for ‘others’ as a reason, too constrained within fixed categories

à Subcategorisation. Creating new main categories.

Page 35: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

35 © 2012 JIRO’S AWESOME NOTES

10 – Experiments Experiments: manipulating one variable to see if another variable changes as a result. It’s more than just asking (as in surveys), but to try them out in practice. 10.1 Experiment to determine Causality To see if variables have a real causal relationship. Does A (independent) cause B (dependent)? Only if:- Temporal Ordering Covariance Elimination of Ext Factors A must precede B in time A and B must vary together B must demonstrably be

caused ONLY by A and not something else

A must first exist, only then B can be affected

Any changes to A must change B at the same time

Ensure B is not caused by anything else.

à Experiments can capture change in variable thru time (surveys cannot)

à Still not enough to conclude causality (other factors might influence B)

Strength: can identify variables with causal relationships, and find out the direction of causality Weakness: Artificial, mental condition different in experimental states (ecological isomorphism) à don’t resemble real life 10.2 Solomon’s 4-Group Design for 2 Variables Formulates RQ à 2-tail hypothesis.

One-group Pretest-Posttest

Design

O1 X O2

1. O1: Baseline observation 2. X: Exposure to an experimental

condition 3. O2: Post-experimental obs

à Does not take into account other variables that may play a part à Need a control

Control (groups) • To remove all other possible variables from experimental design • To be sure only treatment variable itself is causing change • Control Groups: Not exposed to any experimental variable

Two-group Pretest-Posttest

Design + Control

O1 X O2 O1 O2

• Use 2 groups, but place only 1 group in a control environment (isolate it à no treatment variable)

à If control group changes à something else is causing change à Still cannot conclude, since characteristics in one group (as compared to another) may cause the results à need random assignment

Page 36: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

36 © 2012 JIRO’S AWESOME NOTES

If more than 1 characteristic in a group à factors other than experimental variable could cause the change. Cannot remove these variables cos we don't know what they are. Random assignment: equal chance of any peculiarity appearing in both experiment and control groups. Random assignment =/= random sampling. Since unknown factors now influence both groups equally, we can eliminate them out.

Two-Group Random

Assignment Pretest-Posttest

Design R O1 X O2

• Same as 2-Grp Pretest-Posttest • Individuals are now randomly assigned (R)

+ Control R O1 O2

à All other factors can occur equally in both groups à Any changes resultant must be reasonably assumed as due to experimental variable

Non-pretest 3rd (Ctrl) Group

R X O2

• to eliminate possibility of pretest results O1 affecting posttest results O2 à ‘O1’ is removed

• Group has no pretest, but will go thru X and be observed at posttest O2

Non-pretest, Non-variable

4th (Ctrl) Group R O2

• Group of randomly assigned individuals who simply has posttest O2

Two-Group Random Assignment Pretest-

Posttest Design

R O1 X O2

+ Control

R O1 O2

Non-pretest 3rd (Ctrl) Group

R X O2

Non-pretest, Non-variable

4th (Ctrl) Group

R O2 Test Group à What will happen

w/o exp var? à Make sure pretest

does not influence results

à Make sure only pretest and variable influence experiment

Solomon Four-Group Design è To determine specific influence of a variable Time-series Analysis: to see if results will hold thru time. R O1 O2 O3 O4 X O5 O6 O7 O8

Page 37: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

37 © 2012 JIRO’S AWESOME NOTES

10.3 Factorial Designs for > 2 Variables Factorial Designs: for more than 2 variables (3 or more) Formulates 2-tail hypothesis. e.g. 2 x 2, 2 x 2 x 2 10.4 Between Subjects vs Within-Subjects Design To reduce no. of participants (between subjects), we can reuse them again (within-subjects design). Problem 1: One experimental condition will affect the other condition. E.g. Being previously in a group study session will affect their reaction to individual study. Problem 2: Not possible to be both sometimes. E.g. Cannot be guy and girl at the same time. 10.5 Validity and Experimental Design Some variables of interest can be more easily operationalized than others. E.g. Gender à Ask students to reply (Male / Female) Other variables are subjective / difficult to define. E.g. Being ‘In a study group’ à Online study group? Active or inactive participation? E.g. ‘Test scores’ à Can numerical scores capture true academic performance outside tests? Also, require many participants à Impractical

Page 38: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

38 © 2012 JIRO’S AWESOME NOTES

11 – Content Analysis: Understanding Text & Image Media content analysis includes any recorded communication. Content analysis: a quantitative, systematic and objective technique for describing manifest content of communications. Quantitative Systematic Objective Manifest Count occurrences of whatever we're interested in

Count all relevant aspects of sample; cannot arbitrarily pick what aspects get analyzed

Select units for analysis, categorize them using clearly defined criteria

Count the tangible, & observable (the opposite of abstract and latent)

Latent vs manifest content: latent deals with measuring a concept, manifest deals with measuring variables that operationalize a concept. E.g. Cannot count ‘patriotism’ as a concept because it’s latent / abstract. But we can count the frequency of the word ‘patriotism’.

11.1 Advantages of Content Analysis Quantitative emphasis • Emphasis on systematic coding,

counting and analysis of content

Verifiable results • If procedures are explicit, precise and replicable

Availability • Raw materials (things to be studied) are readily available

Cheap and doable • Can be done anytime • Low cost

Don’t need approvals • Human research approvals not required

11.2 Disadvantages of Content Analysis Addresses only questions of content

• Cannot determine causality

E.g. Have public sentiment of the govt improved since GE?

Usefulness limited to comparisons

• Only has application if used for comparisons

E.g. frequency of ‘patriotism’ used by Candidate X MUST be compared with Cand Y to have meaning

Assumptions cannot be made easily

• Interpretation can be subjective / inaccurate

E.g.: Just because Cand X used the word ‘patriotism’ more often, doesn't mean he is really more patriotic

Page 39: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

39 © 2012 JIRO’S AWESOME NOTES

11.3 Seven Steps to a Content Analysis Study Good content analysis requires: systematic sampling, clear definition of units and unambiguous categories. HQ à Universe à Sampling Frame à Sampled Unit

1. Develop H/RQ about comm. content

E.g., H1: Alcohol and tobacco products will differ from pharmaceuticals in frequency of occurrence in comic strips.

2. Define universe of content to analyze

• Decide how many different instances of a unit to be included in the ‘universe’

• Can narrow down the scope of universe

• Universe à narrow down àsampling frame (list from which to draw specific samples)

à comic strips, wherever they come from - books, websites, newspapers à Refine by deciding, say, you only want strips featuring human characters

3. Sample universe of content

• This is narrowed universe à the sampling frame • Define how frequent is the systematic sampling • E.g. yearly systematic sampling, monthly sys sampl. •

4. Select units for coding (5 types) Decide what units to sample

• Physical: occupy observable space in print media or time in audiovisual media.

• Syntactical: units of language

• Referential • Propositional • Thematic

E.g. 1 entire comic strip or individual panels in a strip? E.g. words, sentences visual content, wording à Refer to a person/event à Structures stories and dramas à Broad topics within a structure

5. Develop coding scheme

• Coding sheet-recording frequencies of appearance. • Categories cannot overlap and no unit can be coded twice • For things that don't fit anywhere, might have to create

generic category for them. 6. Assigning

occurrences of a unit to a code in coding scheme

• Identify word references • Assign each occurrence of

a unit in the sample to a code in coding scheme

• For any analysis involving multiple coders, code a no. of units as trial run.

E.g. “Beer”, “Aspirin”, “Cigar” refers to ‘Substance’

7. Count occurrences & report their frequencies

• Not very informative, so have to supplement with additional information like conditions and types.

• Extra info goes into an additional column on coding sheet.

Chi-square test tells us whether distribution of occurrences varies significantly by type of user or by setting.

Page 40: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

40 © 2012 JIRO’S AWESOME NOTES

11.4 Other forms of Analysis Interaction analysis

• Capture and understand interactions among group members and the roles they play

• Behaviors can be categorized as:- o Task-oriented (focus on group’s work by assigning tasks) o Group-oriented and (ensure group remains cohesive) o Self-centered (refuse to participate or dominate discussions)

• Group becomes the unit of analysis • The observed behavior is coded

Conversation analysis

• Looks for rules governing social acts, studying the mechanisms that allow conversations to happen successfully

• Interested in mechanisms that ensure a convo takes place successfully

• NOT about content of conversation • Adds convo-related codes (e.g. time

between utterances)

à Convo transcripts examined for:-

• Turn taking (speaking rights handed from one person to another),

• Adjacency pairs (Q&A)

• Repair mechanisms (actions that restore a conversation threatening to break down)

Rhetorical and dramatistic analyses

• Examine texts to identify and assess their persuasive strategies • (Aristotle) Rhetorical à determine effectivensss :-

o Ethos – character of the speaker o Pathos – use of emotion o Logos – use of logic and facts

• (Burke) Dramatistic pentad (five-part) analysis:- o Act (what) o Agent (who) o Agency (how) o Scene (where, when) o Purpose (why) by answering first 4 questions, you will get

the answer for 5th Ratio analysis

• Examining relative significance of each pentad unit in a given situation

E.g. act-scene, act-agent, act-agency --> gain insight into motives behind comm and reveal inconsistencies between elements

Semiotics analyses

• Concerned with relationship between language, esp. signs and their meanings to us

• Since relationship is arbitrary, multiple interpretations are possible

• Researcher explores possibilities for misinterpretation and so becomes alert to nuances and possibilities of interpretation

E.g. “Stop” sign, yet we take it as ‘Stop, and proceed when it is safe’ E.g. Some people take Stop sign as ‘drive a bit slower if you don’t see the police’

Narrative analyses

• Analyzes formal properties of stories. Generally attempts to identify plot, setting, characters and order of events. Specific attention to how stories play out over time.

Page 41: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

41 © 2012 JIRO’S AWESOME NOTES

Discourse Analyses

• Focuses on systems of meaning and how particular labels/concepts are developed and strengthened through use of language

Critical analyses

• Ways in which communication establishes, reinforces and maintains power structures in society.

• Marxist criticism à examine content for hidden messages that reinforce ideology of system (e.g. management in a company)

• Feminist criticisms à assess content from a woman's perspectives Purposes and methods for analyzing media content often overlap.

Page 42: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

42 © 2012 JIRO’S AWESOME NOTES

12 – Writing Research & Review Research papers: detail and discuss the theory behind the research, the research method(s), sampling, results and conclusions. 12.1 Writing Process 2 extreme ways to write:-

• Imagine the entire ‘final product’ and simply write it down

• Before the research even begins • Only lack the results and conclusions

Or, write and revise by critical examination and multiple re-workings

à necessary for standard survey and experimental designs à need to make known beforehand the Hypotheses / RQs, identify people sampled, methods and review literature

à necessary for projects have no initial hypotheses / RQs or specific research tools à group write copious notes à arranges notes into categories as research progresses à thinking about the project itself shapes the final paper

Most research fall between the 2 extremes. We should have our basic stuff like literature reviews, basic RQs and a sampling description etc. 12.2 Importance of Writing Scholarly Use standard strict writing styles (e.g. APA, MLA, Chigago) to ensure all relevant aspects of research are appropriately reported. Citations must be accurate:-

• Help other researchers access the material too • Give the sources due public recognition • Avoid plagiarism

12.3 Paper vs Panel Can be ‘made public’ thru scholarly paper (publishing reports) or panel presentation (in academic conferences). è made for scholarly public (the people who are in academic study)

Page 43: JAN CS2008 Fundamentals of Research Ch1-14 Complete

CS2008 – Fundamentals of Research

43 © 2012 JIRO’S AWESOME NOTES

12.4 Importance of Writing for Other Publics Interest Groups News Media e.g. Govt corps, NGOs, Interest groups, Agencies à a Professional report is presented to clients, since its written for professionals

X Lit Review X Reference X Jargons V Basic Intro-Body-Conc V Gives audience idea of what the rsher thinks à need to adapt scholarly writing to the wide public audience so they can understand à results, conclusion & relevance

à answers ‘WIIFM’ (what’s in it for me) when writing à need to be timely and relevant à news format, news style

e.g. a study on how TV ads can influence food preferences to children à interest parents, regulatory agencies etc

e.g. a study on cross-cult comm à a story on how to date someone from another culture

12.5 Ethics of Style and Accuracy Writing objectively (in a passive voice) vs. impressively (using the active, 1st person voice). Writing knowingly or otherwise shape readers’ interpretation of research with choice of words. Details can be lost in summary reports or news writing of research. How to give readers unbiased reporting of our research?

• Systematic reporting of RQs, lit reviews, methods, sampling decisions will help scholars decide

• Peer review process ensures published papers met scholarly standards • Instead of trying eliminating all biases, we make them explicit