Beck - Overview of Educational Research HANDOUT · 5/5/2009 2 Compare inductive and deductive approaches to research. Identify when to use an inducti ve versus an deductive approach

5/5/2009

1

Diane E. Beck, Pharm.D.

Interim Associate Dean for Curriculum and AssessmentInterim Associate Dean for Curriculum and Assessment & Professor

College of Pharmacy University of Florida

Introduction

Research that uses a systematic

h d l l i

2

methodology to explain educational problems and issues.

5/5/2009

2

Compare inductive and deductive approaches to research. Identify when to use an inductive versus an deductive approach.

Explain the term “causation” Describe why research designs that investigate causation are

“confirmatory”.

Describe the common educational research designs. Discuss when it is appropriate to use the design and skills needed

b th h t th d iby the researcher to use the design.

For Projects Proposed by Med Ed Fellows: Identify the type of design needed and “why” the design is

appropriate.

Identify attributes of effective curricular resources.

Goal:Identify irrefutable facts & i l t th

Deductive Inductive

Goal:New insights from data ll t d/ b ti& universal truths

Make Predictions Based on General

Principles

Drawgeneralizations from a limited

collected/observations

4

Common in Educational Research

Typical of Clinical/ Laboratory Research

number of observations

5/5/2009

3

Deductive Inductive

Develop a Conceptual Framework via Literature

Search5

Probable Cause

EffectCause

Independent Variables

Dependent Variables

Explained by one of the following:• Theory: Connected hypotheses and variables identified by

the authors. (Explains & Predicts)• Conceptual Framework: Visual model described by other

6

authors; structure from their “lens”• Theoretical Rationale: Rationale proposed by other

authors based on studies about relationships• Hunch

5/5/2009

4

Quantitative Research Designs

Qualitative Research Designs

Combined Qualitative and

Quantitative D iDesigns

Deductive

Inductive

Both

Investigators ask a narrow, specific question, collects quantitative data, and uses statisticsto analyze data.

Investigators ask broad, general questions, rely on the views of participants, collects data consisting of “words”, and analyzes the data for “themes.”

Both quantitative and qualitative methods are used

Is the research confirmatory?

Do you want to evaluate

Causation?

Use Inductive/

Qualitative methods

NoYes

Can you randomize?

Use randomized

Use quasi-experimental

Use correlational

methods

Example 1: e planator

Example 1:Interviews;

focus groups

Example 2:

Yes No

NoYesNote: If you want to use a quasi‐experimental method, but can not assign subjects to a control group, a do ed

experimental methods

Example: RCT

experimental

Methods*

Example: Pre-test post-test

explanatory correlations

Example 2:

Surveys

a p eObservation

Example 3: Surveys – open

ended questions

8

g p,use “non‐experimental methods” such as case‐control, cross‐sectional, time series, and cohort studies.

Deductive Inductive

5/5/2009

5



Causation?

Use Inductive/

Qualitative methods

NoYes

Can you randomize?

Use randomized


Use correlational

methods


E xample 1:: Interviews;

focus groups

Example 2:

Yes No

NoYes

randomized experimental

methods

Example: RCT

experimental

Methods*



Example 2:

Surveys

a p eObservation


ended questions

9

Note: If you want to use a quasi‐experimental method, but can not assign subjects to a control group, use “non‐experimental methods” such as case‐control, cross‐sectional, time series, and cohort studies.

“The Gold Standard” if you want to evaluate “causation” Confirmatory

When to Use: You have a hypothesis to test (supported by a

theory, etc). You can randomize subjects to treatment andYou can randomize subjects to treatment and

control groups.

Skills Needed by the Researcher: Quantitative Research Methods Statistics

5/5/2009

6

The Most Common Design in Education RCT not possible when there are logistic issues/not

practical to randomize

Examples: ○ Non-equivalent control groups

○ Pre-test post-test design

When to Use: You have a hypothesis to test (supported by a theory, etc).

You can NOT randomize subjects to treatment and control groups.

Skills Needed by the Researcher: Quantitative Research Methods

Statistics



Causation?

Use Inductive/

Qualitative methods

NoYes

Can you randomize?

Use randomized


Use correlational

methods



focus groups

Example 2:

Yes No

NoYes

a do edexperimental

methods

Example: RCT

experimental

Methods*



Example 2:

Surveys

a p eObservation


ended questions

12


5/5/2009

7

Examples: Frequently used to correlate academic success with

admission criteria.○ Does not involve an intervention

When to Use: You want a confirmatory approach but DO NOT want to

prove causation

When you want to show how variables are “associated”

Skills Needed by the Researcher: Quantitative Research Methods

Statistics



Causation?

Use Inductive/

Qualitative methods

NoYes

Can you randomize?

Use randomized


Use correlational

methods



focus groups

Example 2:

Yes No

NoYes

a do edexperimental

methods

Example: RCT

experimental

Methods*



Example 2:

Surveys

a p eObservation


ended questions

14


5/5/2009

8

Use in Education: Frequently used in education; does not involve an intervention.

Usually hypothesizes variablesy yp

When to Use: When you DO NOT want to evaluate causation

When you want to document variables and their relationships

Surveys can also be “exploratory” – esp if open ended questions are used

Skills Needed by the Researcher: Survey Design – the survey is your “tool”; validity is essential

Quantitative Research Methods

Statistics

A survey involves more than giving subjects a “questionnaire”questionnaire See Recommended

References and seek an expert on item writing

5/5/2009

9



Causation?

Use Inductive/

Qualitative methods

NoYes

Can you randomize?

Use randomized


Use correlational

methods


Example 1:Interviews;

focus groups

Example 2:

Yes No

NoYes

randomized experimental

methods

Example: RCT

experimental

Methods*



Example 2:

Surveys

a p eObservation


ended questions

17


Use in Education:A. Interviews (focus groups, individual interviews); observation; think

aloud; and simulated recall are methods used to gather information or “observations” from individuals to identify related factorsor “observations” from individuals to identify related factors.A. Grounded theory research

B. Ethnographic Research

C. Narrative Research

When to Use: A. Little is known about the problem – “Exploratory”

B There is a need to understand what factors or variables contributeB. There is a need to understand what factors or variables contribute to the problem or issue.

C. Note: These are not confirmatory

Skills Needed by the Researcher: Qualitative research skills : In-depth recording, analysis of words using

special software & triangulation of data; Inductive thinking abilities

5/5/2009

10

Explore the life of an individualBiography

PhenomenologyUnderstand the essence of

experiences about a

Inductive (Qualitative) Research Designs

Ethnographic Explore the shared l f

Grounded Theory Research

Phenomenology experiences about a phenomenon (eg, learning)

Develop a theory grounded in data

from the field

Case Study

g pResearch culture of a group

of people

Develop an in-depth analysis of a single case

or multiple casesRefs: Creswell JW. Qualitative Inquiry and Research Design, 1998 and Educational Research 3rd Ed, 2008



Causation?

Use Inductive/

Qualitative methods

NoYes

Can you randomize?

Use randomized


Use correlational

methods



focus groups

Example 2:

Yes No

NoYes

Sometimes Both Inductive and Deductive Methods are

Inductivedeductive

a do edexperimental

methods

Example: RCT

experimental

Methods*



Example 2:

Surveys

a p eObservation


ended questions

20


Used in the Same Project

5/5/2009

11

Use in Education: A combination of inductive and deductive methods are

used such as:○ Begin with an inductive design and then do a deductive designg g g

○ Begin with a deductive design and then do an inductive design

When to Use: Methods are usually not used for evaluating “causation”

Used when you want to identify and document variables and their relationships

Surveys can also be “exploratory” – esp if open ended questions Su eys ca a so be e p o ato y esp ope e ded quest o sare used

Skills Needed by the Researcher: Survey Design – the survey is your “tool”; validity is essential

Quantitative Research Methods

Statistics

Study Design Issues related to Internal

validity Issues related to External

validity

Instrument Design

Potential “Threats to validity “ in Educational Research include:• History• Maturation• Mortality• Selection

22

Instrument Design Issues related to Construct

validity

Data Analysis Issues related to Statistical

Conclusion validity

Instrument Design is very important in educational research

5/5/2009

12

Notes about IRB Approval: Always safer to submit – even if it will be

“exempt”

Go the easy road…..Many institutions with a Health Science Center have an IRB office that handles medical research and another that handles “othermedical research and another that handles other research.”

○ At UF, submit to IRB2 if your project does not involve data that falls under HIPAA: http://irb.ufl.edu/irb02/

23

Envision, Plan, Implement, and Write your project so that it will be accepted by Editors. Start now…….“Do it the right way”

24

5/5/2009

13

Common reasons for rejection in RIME proceedings: Problem statement - 17%

Relevance – 5%

Research Design – 6%

Sample and Sampling – 10%

Instrumentation and Data Collection – 14%

Results – 20%

Discussion and Conclusion – 3%

Title – 3%

Abstract – 2%

Writing/Presentation – 9%

25Bordage G. Academic Medicine. 2001;76;889-896.

26

5/5/2009

14

(Important if your Intervention involves Curricular Resources)

Develop resources based on “How People Learn”Learn” Reference: Bransford JD, Brown AL, Cocking RR (eds). How People

Learn: Brain, Mind, Experience, and School. National Academies Press, 1st Ed., 2000. Available at: http://www.nap.edu/openbook.php?record_id=9853&page=R1

Good URL to start: http://www.vanderbilt.edu/cft/resources/teaching_resources/theory/HPL.htm

Other URL: http://www.newhorizons.org/neuro/neu_review_bransford.htm

See Instructional Design Guides at: http://ets.tlt.psu.edu/learningdesign/

5/5/2009

15

1. Title Page with Abstract (150 words) Abstract should succinctly outline: specific aim

(research question and hypothesis), background/ significance experimental design and methodssignificance, experimental design and methods.

2. Specific Aims

3. Background Literature review that provides the background and

convinces the reader that the project is important/addresses a relevant problem. p p

4. Preliminary work – if applicable If you have already done work in this area, describe

it.

See Next Slide….

5. Experimental Design/Methods Research design, population & sample, power

Continued from Prior Slide

calculation – if possible, intervention, procedures, measurable outcomes, instrumentation (include validity & reliability), data collection & quality control, data analysis

6. Timeline for Proposed Activities

7 Literature Citations7. Literature Citations

8. Appendix IRB approval - Documents for IRB submission

5/5/2009

16

Establish a “conceptual framework” and determine whether inductive and/or deductive approaches are most appropriate.most appropriate.

Select the best design given practical considerations; if using confirmatory designs, anticipate “threats to validity.”

Instruments (eg. exams, surveys) are your “tools” make s re the ha e e idence of“tools”….make sure they have evidence of “validity.”

Before You Begin….Learn reasons why reviewers accept/reject educational manuscripts. Essential Reading: Academic Medicine 2001 Issue

1. Core Reference: A. Creswell JW. Educational Research: Planning, Conducting, and

Evaluating Quantitative and Qualitative Research. Merrill P ti H ll U S ddl Ri NJ 2008 ISBN 0 13Prentice Hall, Upper Saddle River, NJ, 2008, ISBN-0-13-613550-1.

B. Trochim WMK. Web Center for Social Research Methods. (See Knowledge Base link to purchase as a Text). Accessed August 1, 2008, http://www.socialresearchmethods.net/

2. More In-depth References:A. Campbell & Stanley, Experimental and Quasi-experimental

Designs for Research, 1963, Houghton-Miflin, Boston, MA, SISBN 0-395-30787-2

B. Cook and Campbell, Quasi-experimentation: Design & Analysis Issues for Field Settings, 1979, Houghton-Miflin, Boston, MA, ISBN 0-395-30790-2 3-11600

C. Shadish, Cook and Campbell, Experimental and Quasi-experimental Designs for Generalized Causal Inference, Houghton-Miflin, Boston, MA, ISBN-13 978-0-395-61556-0.

32

5/5/2009

17

3. Survey Research: A. Draugalis JR, Coons SJ, Plaza CM. Best practices for survey research

reports: A synopsis for authors and reviewers. Am J Pharm Educ. 2008;72:Article 112008;72:Article 11.

B. Fowler FJ. Survey Research Methods. 3rd ed. Thousand Oaks, CA: Sage Publications, 2002.

C. Scheuren F. What is a Survey? Available at: http://www.whatisasurvey.info/ Accessed February 1, 2009.

D. American Association for Public Opinion Research. Best Practices for Survey Research and Public Opinion Research. Available at: http://www.aapor.org/bestpractices Accessed: February 1, 2009.p p g p y ,

E. Doyle JK. Introduction to Survey Methodology and Design. Available at: http://www.sysurvey.com/tips/introduction_to_survey.htm Accessed: February 1, 2009.

33

4. What Editors Look for: A. Review Academic Medicine, September 2001

Issue (free access):

http://journals.lww.com/academicmedicine/pages/toc.aspx?year=2001&issue=09000

5. Development of a Conceptual p pFramework is “key to success”:

A. Reference: Bordage G. Conceptual frameworks to illuminate and magnify. Medical Education 2009: 43: 312–319.

34

5/5/2009

18

Thank You !!!You !!!

Diane E. Beck, Pharm.D.

[email protected]

Phone: 352-273-6279

University of Florida

College of Pharmacy