Public Economics

Chikako YamauchiAssistant Professor, GRIPS

Lecture 2

Tools of Positive AnalysisRosen & Gayer, Ch. 2

Todays topics

1. The Role of Theory2. Causation versus Correlation3. Experimental Studies4. Observational Studies5. Quasi-experimental Studies

1. The Role of Theory

Why is it so hard to tell whats going on?

The Bush administration reduced the income tax rate for high earners

In the 2008 election, John McCain supported keeping it; Obama did not

Conservatives think that lower tax rates provide incentives for people to work harder

Liberals think that tax rates do not matter

Why is it so hard to tell whats going on?

What is the impact of tax rates on the number of work hours?

More generally, how can we estimate the impact of government programs on individual behavior?

The Role of Theory

Theory provides a framework for thinking about the factors that might influence the behavior of interest

For example, suppose Roger derives utility from leisure, but he needs to earn income to buy goods

Suppose his wage rate is $10 per hour, and he finds a combination of income and leisure with which he feels the happiest

The Role of Theory

Now suppose that 20% tax is imposed, and his net wage rate is $8. What happens?

Substitution effect: leisure is cheaper, so consume more of it, i.e., work less

Income effect: he earns only $8 by working one hour, loss in income induces him to consume less of goods and leisure, i.e., work more

The Role of Theory

Thus, theory indicates the mechanism through which a policy might affects individual behavior

Sometimes it includes conflicting effects such as substitution and income effects

Under such circumstances, only empirical work can tell us the overall effect of a policy on individual behavior

2. Causation versus Correlation

Causation versus Correlation

To establish a causal relationship, The cause (X) must precede the effect (Y) X must be correlated with Y Other explanations for any observed

correlation must be eliminated

The last condition is difficult to pass

Causation versus Correlation

Suppose we are interested in the effect of unemployment insurance (UI, payments to people who lost jobs) on unemployment spell

Those who received high benefits = treatment group

Those who received low benefits = control group Suppose the treatment group exhibited a shorter

spell of unemployment subsequently

Causation versus Correlation

It could be because UI had an impact to shorten unemployment spell

However, a possible other explanation is that the treatment group were different in other ways High UI benefits are typically given to people

who had higher earnings in their previous jobs They might have greater motivation for work

3. Experimental Studies

Experimental Studies

The observed relationship between UI benefits and unemployment duration was due to a third influence, i.e., motivation level

Thus, the lower unemployment duration for the treatment group relative to the control was a biased estimate of the true causal impact of the higher benefits

Experimental Studies

To rule out other factors, we want to know the counterfactual: what would have happened to members of the treatment group had they not received the treatment

Experimental Studies

Golden standard = experimental study: a study in which individuals are randomly assigned to the treatment and control groups

Since the selection into the treatment group is out of individual control, it is less likely that factors, such as motivation level, differ between groups on average

Conducting an Experimental Study

First, randomly assign a sample of unemployed people to receive high or low benefits

Second, check whether observed characteristics (age, education, gender) are similar on average

Third, compare the subsequent average unemployment duration

Pitfalls of Experimental Studies

Ethical problem: can we force a certain group of people to be exposed to harmful treatment to measure its effect? pollution

No compliance: the treated may not participate, and the controlled may try to sneak in No compliance among the treated = Effect of Intent

to Treat No compliance among the controlled dilutes the

estimated effect of treatment Bias towards zero

Pitfalls of Experimental Studies

Nonrandom attrition: disappearance of certain members from data Suppose we are interested in the effect of job-training

program on future wage rates Suppose the program actually increased the wage of

participants Suppose also that low-skilled workers in the control

group did not experience an increase in the wage and felt ashamed. They may not report their wages to researchers

This increases the average wage rate only among the controlled

The researchers may wrongly conclude that the treated and the controlled have the same wage rate

Pitfalls of Experimental Studies Scope for generalization: true effect may differ

when a program is expanded indefinitely instead of temporarily

Effect of generous health insurance on frequency of doctor visits

If the treated receives generous health insurance for a year only under an experiment, they may increase doctor visits because they know that they will lose it next year

at the national, not local, level Effect of completing a college degree on the wage rate Suppose there was a shortage of college graduates initially New trained college graduates will be highly paid However, if the majority become college graduates, their

supply might exceed demand Then the effect is unlikely to be as high as the local

experiment suggests

Experimental Studies Are Not Foolproof

Mechanism through which X resulted in Y is not clear This returns us to the role of theory. By assuming that

people rationally maximize utility, theory can help us explain particular experimental results, and generalize them to other contexts

Researchers have to check if original randomness is maintained and must be cautious about generalizing the results to other settings

4. Observational Studies

Observational Studies

Empirical studies can rely on observed data, which are not obtained from an experimental setting called observational studies

Why observational studies? Ethically or politically difficult to conduct an

experiment It is difficult to provide treatment without making

subjects aware that they are being evaluated once aware, they might change behavior to create the

outcome they want E.g., in the experiment to examine the effect of tax cuts on

labor supply, subjects might work hard only during the experiment to get tax cuts

Observational Studies

Sources Telephone surveys of consumers Written surveys submitted by households Administrative records on economic

performance, demography, crime, etc.

Econometrics Statistical tools for analyzing economic data Regression analysis is used

Conducting an Observational Study- Suppose that we are interested in the effect of a reduction of the income tax on annual hours of work, L

-Is there an observed correlation between changes in net wage rate, w, and changes in L?

-Independent variable = w

-Dependent variable = L

-Suppose we have data on the hours of work and the after-tax wages for a sample of people for a given year

-We can draw a scatter diagram looks like there is a positive correlation

Conducting an Observational Study-How can we estimate the magnitude of the positive relationship?

-In regression analysis, we attempt to fit a regression line through these points

-The slope of the line is the regression coefficient, which indicates the relationship between w and L

-If the regression coefficient is 1.5, this suggests that an increase in the net wage by $10 is associated with an increase in labor supply by 15 hours per year

Conducting an Observational Study-How do we judge whether the estimated coefficient is reliable?

-This regression line is identical with the previous one, but drawn through the scatter of points that is more diffuse -> less faith on the estimated coefficient

-In Econometrics, such reliability is measured by comparing the size of the coefficient to its standard error (SE): a statistical measure of how much an estimated coefficient might vary from its true value

-Coefficient is reliable if SE is small relative to coefficient

Type of Data

Cross-section data Contain information on entities such as

individuals, firms, and countries at a given point in time

Time-series data Contain information on an entity at different

points in time Panel (longitudinal) data

Contain information on entities at different points in time

Pitfalls of Observational Studies It is difficult to ensure that the control group forms a valid

counterfactural For example, the positive correlation between the net wage and

the hours of work may arise because highly ambitious people have higher wages and also work longer hours

In this case, we overestimate effect of net wage on work hours We might be able to hold the effect of other observed

characteristics constant in multiple regression analysis However, we may not be able to think of all the factors

which affect the hours of work, and may not be able to measure some of them

Nevertheless, observational studies are informative about possible causal effects. We just need to be careful in interpreting the results as there might be outside factors that might bias any causal inferences

5. Quasi-experimental Studies

Quasi-Experimental Studies

Experimental studies are good at eliminating bias, but difficult to perform

Observational studies have knotty problems with bias, but the data are easier to obtain

Quasi-experimental studies use observational data, but rely on circumstances outside of the researchers control that naturally lead to random assignment natural experiment

An Early Example

The effect of water quality on cholera incidence by John Snow (1855)

Two water companies supplied water to households in London

One had its intake point upstream from the sewage discharges, while the other had it downstream from the discharges

Snow showed that households receiving water from the latter are more likely to be cholera victims

He also showed that households receiving water from the two companies do not differ much

Conducting a Quasi-Experimental Study

Difference-in-Difference Instrumental Variables Regression Discontinuity

Difference-in-Differences

Suppose we are interested in the effect of raising the tax on beer on teen traffic fatalities

A group of U.S. states increased their tax rates on beer between 1989 and 1992, and teen traffic fatalities declined by 5.2 per 100,000 teens

Can we use this information to conclude that higher taxes on beer lower teen traffic fatalities?

No! teen traffic fatalities might have lowered without the tax rate change

We can compare changes in teen traffic fatalities in states which did not change the tax rate on beer

Dee (1999) found that in the control group the fatalities declined by 8.1 per 100,000 teems

Thus, the tax increases did not reduce teen traffic deaths

If we can assume that the control group provides a valid counterfactual for the trend/change, this Diff-in-Diff achieves unbiased results

Difference-in-Differences

Instrumental Variables Method

Sometimes assignment into a treatment group may not be random E.g., states that did not introduce beer tax

might have had a increasing state budget, which enabled better infrastructure, thus reducing accidents

If trends differ, we cannot use DD IV = variable that affects entry into the

treatment group, but in itself is not correlated with the outcome of interest

Instrumental Variables Method Suppose we are interested in the effect of class

size on childrens test scores An experiment to investigate this issue would

randomly assign students to different class sizes Done by Krueger (1999), but the assignment was only

temporary. Unclear if the results reflect the true effect An observational analysis might use regression

analysis to estimate whether students in smaller classes score higher than students in larger classes

However, what types of parents choose to send their children to schools with small class size? Bias is likely to be an issue

Instrumental Variables Method

Hoxby (2000) observed that kindergarten class size varies across years because the timings of births fluctuate randomly

The random fluctuations in enrollment year-to-year is correlated with class size, but does not directly influence test scores. Thus, it satisfies the conditions to be an instrumental variable

Based on this strategy, she found that class size does not have a discernible effect on test scores

Regression Discontinuity Eligibility for some policy programs is determined by

whether a measurable characteristic of a person is above or below a specific cut-off point E.g., public insurance is available to people whose incomes are

below $20,000 People who earn more than $20,000 and people who

earn less than $20,000 are likely to be not comparable However, people who earn $20,001 and people who

earn $19,999 are likely to be similar to each other RD relies on such a strict cut-off criterion for eligibility of

the intervention under study to approximate an experimental design

Regression Discontinuity Suppose we are interested in the effect of mandatory

summer school for poorly performing students on their test scores

Experiment: most likely politically infeasible Observational studies: most likely the treatment group

(poorly performing students) is not comparable to the control group (well-performing students)

RD: In Chicago Public Schools, students who scored below a cut-off on the test were required to attend summer school

Jacob and Lefgren (2004) compared students who got scores just above and below the cut-off

As a result, they found a jump in follow-up reading and math scores for 3rd graders, but not for 6th graders. This suggests the existence of a positive effect at least for some grade levels

Pitfalls of Quasi-Experimental Studies

It may not truly mimic random assignment to the treatment group DD: the two groups of states might have had

different trends in teen traffic fatalities IV: across-year variation in birth cohort size

might have been correlated with child cognitive skills (not random)

RD: students whose score were just above and below the cut-off might have been different

Pitfalls of Quasi-Experimental Studies

Quasi-experiments approach cannot be used to address questions which involve the national- or global-level changes E.g., if the government does not provide pensions,

would people save more? If the pension system is introduced for everyone at

the same time in one country, there is no control group

How much can we generalize the results based on quasi-experiments? Specific to the intervention being studied Mechanism through which the estimated effect has

arisen is not always clear