Research DesignLoyola Law School – Fall 2007

Doug Stenstromemail: stenstro@usc.eduphone: (213)422-0909

http://www.psychwiki.com/wiki/Research_Design_Fall_2007

Today’s objective

• Overview of the Research Process• Help you map out your semester (e.g., time, tasks,

etc.)

• Answer questions you may have about design/process

• Talk about the issue of collaborations, authorship, etc.

• If time – Introduction to the field of Psychology

Purpose of the Course

After completing this course, you will be able to:

• Critically examine claims made about human behavior • Design and conduct all types of research studies • Evaluate research validity – Construct, Internal, External,

Statistical

• Write a research proposal – Introduction, Method, Discussion, Results

• Learn how to think like a research psychologist • Comprehend and effectively use the vocabulary

terminology of research design

Why Research Design is important:

• As a Researcher…• As a Consumer…

Why Scientific Method is important:

• Most important advance in science in last 1000 years

• Why? Moving away from mere observation and common sense, to scientifically evaluating “reality” via data collection, statistics, interpretation, dissemination

Overview of the Research Process

AnalyzingData

CollectingData

ResearchIdea

ResearchDesign

Publication Process

Pick a topicTranslate topic into QuestionTranslate into HypothesisPreemption search

Selecting the methodological approachOperationalizing variables/materialsCrafting procedures/paradigmsDetermining sampleEvaluating the practicality of the research

Approval for study via IRBPreparation to conduct studyRecruiting subjectsConducting the studyCoding and entering data

Data preparation and screeningEvaluating sample statisticallyEvaluating materials statisticallyEvaluating procedures statisticallyAnalyzing research HypothesisInterpreting the results

Deciding when to start writing paperHow to write psychology manuscriptsHow to write in APA formatHow to decide where to submit manuscriptJournal review processTalks, posters, and other ways to disseminate your work

Pre-productionPre-production ProductionProduction Post-productionPost-production

Example – “lay person survey”

Questions:#1 What percent of people

brought to trial did in fact commit the crime?

#2 What percent of people convicted by juries in criminal trials did in fact commit the crime?

#3 How much trust do you have in the accuracy of convictions decided by juries in criminal trials?

#4 In order to convict a person for a crime, jurors should feel that it is at least _____% likely that the defendant is guilty of the crime.

Descriptive:

78.39%

83.32%

66.55%

90.53%

Relational:All questions had a positive and medium size relationship to each other.

Some questions had a relationship to demographics, such as religious people had higher percentage for #1 compared to non-religious, and women had a higher percentage for #4 compared to men.

Example – Eyewitness identification

• Research Idea - Which is the best method of eyewitness identification: Simultaneous lineup or Sequential lineup?

• Research Design– Methodological approach:

• Archival data? Survey data? Experiment data? etc.– Operationalize “identification”:

• 100% accuracy, 90%, 80%, etc.– Operationalize the materials:

• Should the lineup contain fillers? How to determine whether the fillers are “good alternative”? What instructions for witnesses? What order for Sequential? etc.

– Craft the procedures: • Should the “crime” be something the subjects read? See? How to model

“real world” crime situations? etc. – Determine sample:

• Which population to you want to generalize to? From a practical standpoint, how to find a convenient supply of subjects? etc.

– Practicality of the research: • Do you have the money/time/manpower to conduct the ideal study?

What concessions will you have to make? How to decide which concessions are valid and which will reduce the study to being unscientific? etc.

Collaboration – who, how, what

Who:• Undergraduate Research Assistants (RA)• Graduate Research Assistants • Faculty at your university• Faculty at another university• Specialized Consultants

How:• “Lab Group” in-person meetings with all above people• “Virtual” Lab Group online exchanges with outside

people

Authorship:• Undergraduate – typically not• Graduate - depends• Faculty – depends• Specialized Consultants – typically yes

• Pick a topic– RA: search literature, collect ideas– Grad: same– Faculty: email and ask

• Translate topic into Question– Lab Group Discussion

• Translate into Hypothesis– Lab Group Discussion

• Preemption search– RA: search literature for all related

research– Grad: same– Faculty: email and ask

ResearchIdea Collaboration

• Selecting the methodological approach– Lab Group Discussion

• Operationalizing variables/materials– RA: search literature for all relevant research– Grad: same– Faculty: email and ask– Lab Group Discussion

• Crafting procedures/paradigms– RA: search literature for all relevant research– Grad: same– Faculty: email and ask– Lab Group Discussion

• Determining sample– RA: find “power” via books/websites

• Evaluating the practicality of the research– Just you

CollaborationResearchDesign

• Approval for study via IRB• Preparation to conduct study• Recruiting subjects• Conducting the study• Coding and entering data

For all steps above, it falls to you or RA/Graduate– You may want to do it first time– Then pass to RA, Graduate student

CollaborationCollectingData

• Data preparation and screening• Evaluating sample statistically• Evaluating materials statistically• Evaluating procedures statistically• Analyzing research Hypothesis• Interpreting the results

For all steps above, it falls to you or RA/Graduate– You may want to do it first time– Then pass to RA, Graduate student

CollaborationAnalyzingData

• Deciding when to start writing paper• How to write psychology manuscripts• How to write in APA format• How to decide where to submit manuscript• Journal review process• Talks, posters, and other ways to disseminate your work

For all steps above:– RA: RA writes only if give that RA authorship.

RA sometimes gets poster authorship.– Grad: Grad can get authorship even if not writing.

Grad can write whole manuscript or just parts.If just parts, typically Method, Results.If just parts, maybe even first draft of Intro,

Discussion.Grad provides Reviews/Comments.Grad provides Re-writes.Even if not author, Grad can provide

Reviews/Comments.– Faculty: If author, Faculty can write/rewrite Intro,

Discussion.If not author, can provide Reviews/Comments.

CollaborationPublication Process

Best way to learn is by doing…

Recommendations:• Start developing idea(s) now

– Our next class is about generating research ideas– For now, try to identify “topics” – You are welcome to conduct multiple studies this semester

• I would recommend doing a project for this course… … even if you have no research ideas (I can help with that) … even if you have limited time (see short-cuts below)

• If pressed for time, there are some short-cut(s)…– Pre-emption can be skipped because unlikely “exact” idea has been done– Research idea need not be overly complex or complicated– ASAP – Legal topic, frame as social policy, collect basic descriptive

information – Pick research idea by reading published article and adding 1 new aspect– Can “farm-out” collecting data and analyzing results– Collaborate (attach yourself) to psychologist’s existing research– Collaborate (attach yourself) to psychologist but conduct your own idea

Best way to learn is by doing…

Recommendations:• Use this class as “Lab Group”

• Present to the “Lab Group” for…– Phase 1 – Research Idea

• Get feedback about research Question• Get feedback about research Hypothesis

– Phase 2 - Research Design• Get feedback about selecting Methodological approach• Operationalization of variables• Get feedback about Crafting materials/paradigm

– I can provide one-on-one help for other phases• Phase 3 – Collecting Data (I can help solve problems that may arise)• Phase 4 – Analyzing Data (If you want to analyze before Spring’s Statistics

class)• Phase 5 – Publication Process (I can explain appropriate journals,

conferences, etc

• Get RAs– Time investment for 1 RA is the same as 1+ RAs if you instruct them in

groups (e.g., your own Lab Group), so I would recommend getting a few RAs

– RAs can help with more time-consuming tasks like literature search, preemption, finding relevant published articles, etc.