GS/PPAL 6200 3.00 Section NResearch Methods and Information Systems

February 3, 2015

Professor Brenda Spotton VisanoOffice: 130 McLaughlin

Voice Mail: (416) 736-2100 ext. 20470 E-mail: spotton@yorku.ca

Agenda

• Review of last class• Sample size in research – how big is big

enough?• Small-N debate• Medium-N and mixed methods

Review of Last Class• Presentation by David Northrup and Hugh McCague from the

Institute for Social Research/York University • “Ice is cold” - How do we “know” this? Common "ways of

knowing“ include…– Casual empirical observation, Tradition, An authority said so, Religion,

Inductively (from specifics to the general), Deductively (from the general to specifics)

• “Knowing”: Mill’s Methods of Agreement, Difference, Agreement & Difference, Concomitant Variation

• Observations: What’s specific? Cases versus variables• Observations: What’s general? How many observations are

sufficient? • Sampling Types and Strategies for Selection

Sampling Issues (Why Sample?)

• Sample Types: Representative (typical of category), Prototypical (expected to become typical), Deviant (an exception to the norm), Crucial (tests in the least favorable conditions), Archetypical (creates a category)

• Sample Selection: Probabilistic (random, stratified random, systematic random) or Non-probabilistic (convenience, purposive, expert or snowball)

• Sample Size: How many is enough?

Sample Size

• Is the Research Question privileging depth or breadth – internal or external validity? Is the researcher trying to describe a phenomenon in detail (smaller sample), develop a hypothesis about it, or to test it (larger sample)? Should the sample be weighted to reflect population differences?

• Link to Qualitative versus Quantitative Methods of Data Collection and Analysis

• If the researcher uses a smaller sample, can they make “big” conclusions…

Small-N and Big Conclusions The Lieberson – Savolainen Debate

• Why emphasize the “probabilistic” nature of research?

• “The value of Mill’s methods is in their capacity to eliminate a limited set of alternative causal statements.”

• More generally, what is the value of conducting small-N research? What are its limitations? (What’s the tradeoff between large and small sample studies?)

Medium – N Analyses…and Big Conclusions?

• Paul, Clarke, Grill, Savitsky (2013) • Recall the Checklist of Questions: – What is their research question? – What is their sample and how would you

characterize it (type, selection, size)? – How did they collect their data? – How do they analyze it?

Bivariate Relationships

• Factor (A) present/absent – Case outcome (x: win, y: loss) (see Table 2, p.226)

• A BC –> x =5, y = 0• BC –> x =3, y =22• Implicitly then by Mill’s Method of Difference,

there is a probability that the absence of A (COIN force credibility) influences counter insurgency “losses”

And Mill’s methods?

• “It could be that one or more of the three critical factors are not really present in the exceptional case but they were evaluated as present based on a superficial reading of the history, which a detailed review exposes as incorrect.”

• GIVEN: Superficial reading of history, conclude ABC occur together with xyz

• AND: Exceptional case (for one factor not present): BC occur together with xyz

• Can we conclude anything by any of Mill’s methods?

And Mill’s methods? (cont’d)

• “Or it could be that the three critical factors are very much present, but a detailed exploration reveals…the impact of the three factors is thwarted by the presence of a fourth factor,…”

• GIVEN: Superficial reading of history, conclude ABC occur together with xyz

• AND: exceptional case reveals “thwarting factor”: ABCD does NOT occur with xyz

• Can we conclude anything by any of Mill’s methods?

Discovery and Proof

• What causes Counter-insurgency success? • What causes major civil uprisings (revolutions)?• “Mill’s method of agreement is…worthless as a

method of discovery and fallacious as a canon of proof. Its true value is in its function to eliminate alternative explanations…. No factor can explain an outcome satisfactorily that is not common to all occurrences of that outcome.”