The Effects of International Agreementson Preferences and Beliefs

Michael TomzStanford University

IPES, November 2006

When (if ever) do international agreements matter?

• Skeptics: IA’s have little independent effect (e.g. Mearsheimer)

• Legalists: IA’s change preferences and beliefs (e.g. Keohane)

Roadblock to research: selection bias

• Standard approach: compare behavior of signatories versus non-signatories

• Problem: Membership is voluntary, which raises the potential for section bias.

Two potential sources of selection bias.

1. Different baseline propensities, e.g. signatories are “more willing” than non-signatories.

2. Heterogeneous treatment effects, e.g. signatories are “more susceptible” than non-signatories.

)0|()1|()0|()1|(ˆ00 AAAYAY

Different baseline Heterogeneous effects

It’s hard to solve the problem with statistical methods.

1. Control variables

Goal: Make assignment random conditional on XsLimitation: Need a comprehensive set of controls

2. Instrumental variables

Goal: Find variable that affects Y only through IALimitation: Hard to find strong instruments

A solution: embed experiments in voter/elite interviews.

1. Randomly assign subjects to 2 groups.

Group 1: International agreement existsGroup 2: No international agreement

2. Compare the views of the 2 groups.

Directly measures policy preferences/beliefs.Avoids the bias from nonrandom selection.

Example 1: experiment to study policy preferences.

• All consider trade with Burma

• Randomize pro/con argumentsHelp the US economyHelp human rightsHurt the Burmese economyViolate treaties (international law)

• Measure policy preferencesHow good or bad an idea is it for theUS to prohibit trade with Burma?

The experiment was administered to US adults.

• Knowledge Networks ran 1000 interviews.

• The sample was nationally representative.

• The treatment groups were balanced.

Finding: International agreements change preferences.

Effect of int’l agreement = p1 - p0 p1|data ~ beta(b1+1, N1 – b1 +1) p0|data ~ beta(b0+1, N0 – b0 +1)

• I modeled opposition to trade barriers.

• Agreements increased opposition.

Estimate 95% CIAgreement 44 (38 to 51)No Agreement 27 (23 to 31)Effect 17 (10 to 25)

Effect of int’l agreement = p1 - p0 p1|data ~ beta(b1+1, N1 – b1 +1) p0|data ~ beta(b0+1, N0 – b0 +1)

… even in the face of powerful counterarguments

(Table gives % opposition to trade barriers)

Arguments for BarriersHuman rights only 54 (43 to 65) 37 (30 to 45) 17 (4 to 30)U.S. economy only 40 (30 to 51) 30 (23 to 37) 10 (-3 to 23)Both arguments 37 (26 to 49) 15 (10 to 21) 22 (9 to 35)

International AgreementNoYes Effect

The effect diffuses throughout the population.

Liberals 53 (41 to 65) 28 (21 to 36) 25 (11 to 40)Conservatives 47 (35 to 58) 31 (25 to 38) 15 (2 to 29)

Democrats 48 (37 to 59) 27 (21 to 33) 21 (9 to 34)Republicans 43 (32 to 53) 25 (19 to 32) 17 (5 to 30)

Females 45 (37 to 54) 26 (21 to 31) 19 (9 to 30)Males 43 (33 to 53) 28 (22 to 34) 15 (4 to 26)

Some College 47 (38 to 56) 28 (23 to 33) 19 (9 to 29)No College 40 (31 to 50) 26 (20 to 32) 15 (3 to 26)

High Income 46 (36 to 56) 23 (18 to 28) 24 (12 to 35)Low Income 43 (34 to 51) 31 (25 to 36) 12 (2 to 22)

12 (-4 to 27)

-4 (-11 to 20)

5 (-11 to 20)

10 (-10 to 29)

4 (-14 to 22)

International AgreementNoYes Effect Difference

Example 2: experiment to study beliefs in IR.

• All consider nuclear proliferation.

• Randomize available informationRegional security situationStatements of intentionsSatellite images Economic conditionsMembership in the NPT

• Measure beliefsLikelihood that the country is pursuingnuclear weapons?

The experiment was administered to British MP’s.

• Research team conducted face-to-face interviews June-August 2006.

• Sample size was between 60 and 76, depending on the question.

Finding: International agreements change beliefs.

Is country pursuingnuclear weapons? Yes No Effect

Likely 30 63 -3350-50 48 30 18Unlikely 22 7 15

Signed the NPT

In discussion, MP’s cited the NPT as an indication of intentions.

(χ2(2) = 6.4, Pr=.04; τb=.32, se=.12)

Strengths of the experimental approach

1. Overcomes problems of endogeneity

2. Improves microfoundations of IR theory

3. Expands range of answerable questions

Next steps in this research program.*

• Vary issues, contexts, institutional design. What experiments would you like to see?

• Interview policymakers here and abroad. Referrals/connections would be welcome!

• Develop software for experiments. Save money, increase data reliability

(*With funding from NSF CAREER grant)

Each respondent received a set of arguments.

Group U.S. economy

Human rights

Burmese economy

International agreement

Neutral argument

Sample size

1 X X 128 2 X X X 114 3 X X X 125 4 X X 114 5 X X X 103 6 X X X 99 7 X X X 113 8 X X X X 88 9 X X X X 116

Some pro/con points

US economy: “The proposal would help the U.S. economy. Many Americans are getting laid off because of competition from Burma. If we stop trading with Burma, there will be more jobs and higher wages in the United States.”

Human rights: “The proposal would help human rights. In Burma, the government kills political opponents and does not allow free speech. By stopping trade with Burma, we can pressure the government to start respecting basic rights.”

Burmese economy: “The proposal would hurt the Burmese economy. Burma sells $300 million in products to the United States each year. If we stop trading with Burma, people in that country will lose their jobs, and poverty will rise.”

International law: “The proposal would violate international law. The United States has signed treaties that make it illegal to limit trade with Burma. If we stop trading with Burma, we will be breaking international law.”

More pro/con points

I also included a neutral argument.

Neutral argument: “The proposal would change our trade relations. The United States trades with many countries. If we stop trading with Burma, we will no longer suffer the costs (if any) nor will we get the benefits (if any) of trade with that particular country.”

There’s much concern these days about the spread of nuclear weapons. I’m going to describe a country that may or may not be pursuing nuclear weapons. For scientific validity, the description involves a general type of country, rather than a specific country in the news today. Some parts of the description may strike you as very important; other parts may seem much less important. When I have finished the description, I will ask—in your best judgment given the limited information available—how likely or unlikely you think it is that the country is pursuing nuclear weapons.

Nuclear proliferation: introductory script

Nuclear proliferation: available information

• The country borders on an unfriendly nation that has nuclear weapons and has threatened to use them in a future war.

• The country has repeatedly said that it does not want nuclear weapons.

• {The country has signed the Nuclear Non-Proliferation Treaty, thereby pledging “not to receive, manufacture, or otherwise acquire nuclear weapons.”} OR {The country has not signed the Nuclear Non-Proliferation Treaty.}

• Recent satellite images show that the country has started enriching uranium, which could be used for either civilian or military purposes.

• The country has a stagnant economy.

Nuclear proliferation: measure of beliefs

Here’s a card with the scenario I just read, so that you can review the facts.

Now, based on this information, would you say that it is very likely that the country is pursuing nuclear weapons, that it’s somewhat likely, that there’s a 50-50 chance, that it’s somewhat unlikely, or that it’s very unlikely?

Could you explain briefly why you think {ANSWER}?