How Good Are ASEC Earnings Data? A Comparison to SSA Detailed Earning Records.

Joan Turek, Kendall Swenson and Bula Ghose, Department of Health and Human Services

Fritz Scheuren and Daniel Lee, NORC University of Chicago

The views are those expressed by the authors and are not the official position of any of their organizations

2

Acknowledgements

The authors thank Charles T. Nelson, Assistant Division Chief for Economic Characteristics, and Edward J. Welniak Jr., Chief of the Income Statistics Branch, Bureau of the Census, for their extensive assistance in developing the comprehensive data series, including the matched data and for reviewing our findings

3

Research on Effects of Imputation

Past research looking at imputation on the Current Population Survey (CPS), Annual Social and Economic Supplement (ASEC) has focused on:

-Trends -Poverty

-Demographics

4

Current Focus: Match of ASEC/DER

-Joint project by Office of the Assistant Secretary for Planning and Evaluation, Department of Health and Human Services and Census Bureau-Project matches the 2006 ASEC to Social Security’s 2005 Detailed Earnings Records (DER) -Thus, 2005 CY Income

5

Handling CPS Nonresponse

• Imputation occurs at the person level by income source• Assigns amount from reporters with

similar characteristics• ASEC uses “Hot Deck” procedures• Imputation method consistent over

time

6

Income Imputation

• Item impute: sample persons or other household member fails to respond to a specific question. Responses to more than one item may be imputed.• Whole impute: entire supplement is

imputed using the monthly survey

7

Income Imputation (Cont)

• Item imputes use information from the “basic” monthly survey and the ASEC supplement• Limited information on the basic

monthly survey --- have labor force experience for the reference week and not last year

8

Trends in Imputation

• Imputation rates for total dollar income began trending upward after 1988 when the CPS went from 12 to 55 income questions

• Imputation rates rose from 21.2% in 1977 to 35.0% in 2010

• Item imputes have more than doubled since 1987

• Whole imputes have remained fairly stable over this period (1987 to 2010).

9

Percent of Total Dollar Income Imputed for Persons with Positive Income by Year

0%

5%

10%

15%

20%

25%

30%

35%

40%

21.2%

35.0%

10

Percent of Total Dollar Income for Persons with Positive Income by Imputation Type and Year

19871988

19891990

19911992

19931994

19951996

19971998

19992000

20012002

20032004

20052006

20072008

20092010

0%

5%

10%

15%

20%

25%

30%

35%

40%

11.3%

21.8%

9.2%

10.5%

Item Imputes

Whole Imputes

11

Trends in Poverty

• The poverty rates (number of persons in poverty) trend lines for item, whole and no imputes are generally parallel over time, increasing and decreasing with similarity across the years (1987 to 2010

• Poverty rates in any single year differ by imputation status. Persons with no imputes have the highest poverty rate, those with whole imputes are next and those with item imputes have the lowest poverty rate

12

Percent of Persons with Positive Income that are Poor by Type of Imputation and Year

19861987

19881989

19901991

19921993

19941995

19961997

19981999

20002001

20022003

20042005

20062007

20082009

2010

-2%

0%

2%

4%

6%

8%

10%

12%

14%

10.5%11.7%

9.8%9.7%

6.7%

7.2%

No Impute

Whole Impute

Item Impute

13

Impact of Imputation on Poverty Rates for Persons with Positive Income in 2005 (Counterfactual)

% Poor

All Persons with Positive Income 8.59%

No Imputes 10.07%

Item Imputes 6.48%

Whole Imputes 9.14%

Whole Imputes Plus No Imputes 9.91%

Item Imputes Plus No Imputes 8.53%

All Persons with Item Impute Values Set to Zero 25.60%

Item Imputes Only Set to Zero 48.66%

14

The 2005 Calendar Year Census Match Project

• Earnings 82% of ASEC income in 2005• Results are presented for all persons 15 and

older with earnings who matched i.e. - 84.7% of ASEC person records

• Results are also presented for persons in poverty ---DER earnings substituted for ASEC earnings and poverty recomputed. (At least 1 DER matched person in family) – 82% of poverty population (unit counts)

15

Comparing DER and ASEC

• The definition of earnings used in matching the 2005 DER to the 2006 ASEC is:– For The DER: max(Box1Comp+Deferred, Box3Wages) +

max(Medicare_SE, FICA_SE)– For the ASEC: PEARNVAL– Since 1978 DER developed fromW2 tax data

• DER is not a gold standard– Reports SSA covered wages well, but misses some persons

and some sources of earnings

16

How they Differ

• Coverage– Employers may pay some wages underground and employees in many service

industries do not report all of their wages to Social Security, but they may be is ASEC

– Major groups not covered in the DER include civilian federal employees hired before January 1, 1984; railroad workers, certain employees of state and local governments who are covered under their employer’s retirement system; domestic and farm workers whose earnings do not meet certain minimum requirements and persons with very low earnings from self employment. Some of these jobs are covered under Medicare

– Individuals might report different allocations of wage and salary and self-employment earnings from S Corporations and the DER and ASEC

Amy Olsen and Russell Hudson, Social Security Administration’s Master Earnings File: Background Information, SS Bulletin, vol.68, No. 3, 2009, page 41, Footnote 6

17

How they Differ (continued)• What’s Excluded

– DER does not include health insurance premiums that are not taxed under Social Security, Medicare or the Federal Income tax not included in Boxes 1,3,or 5 of the W-2

– Health Savings Account (HSA) exempted compensation is in W-2 Box 12. It is not possible to determine from the W-2 alone or from the MEF how much of HSA shown in W-2 Box 12 is already included in W-2 Boxes 1, 3, or 5 (per IRS consultation). Therefore, if you add HSA to total earnings, as we have chosen to do, you will possibly be overstating the total earnings. If you choose to not add the HSA to total earnings, you will possibly be understating the total earnings. It depends on the sample of workers you are looking at and how common HSA is. Generally speaking, the impact of HSA should be slight

- The same issue may be true regarding 457(b) deferred compensation plans. It is not possible to determine from the W-2 alone or from the Master Earnings File how much of that amount in Box 12 is already included in W-2 Boxes 3 and 5 (per IRS consultation)- W2 can be updated at any time and so some changes are missed

Greg Diez, Social Security Administration, several e-mails.

18

Implications If their employers allow, employees can deduct all of the following pre-tax in addition to their 401(k)

contributions which the DER will give you:

Up to $5,000 for dependent careUp to $5,000 for flexible spending for health careThe cost of their portion of health insurance premiums (this could be in the $5,000 to $10,000 range)Commuting costs which could run in excess of $3000 for people using the maximum.

There’s probably an inverse relationship between dependent care and flex spending: large flex spending deductions tend to come after children are past the need for dependent care. But health insurance premiums will tend to be independent.

The important thing is that these are not defined as proportions of salary the way flex spending is. For a family with $50,000 in income, you could be talking 10 to 20 percent easily. Health insurance premiums alone could be that much. A family with $100,000 in income might be more likely, though, to incur deductible expenses, so the fraction not reported on the W-2 might not decline a whole lot.

John Czajka, Mathematica Policy Research, stated in an e-mail on April 3, 2012

19

Presentation of Results: All Persons 15 and Older with Earnings

• Tabulations were constructed crossing ASEC earnings with DER earnings at $10,000 intervals in order to meet disclosure provisions

• This detailed information collapsed into easier to examine format providing information on how closely results matched—e.g. deviations from same results (within $10,000 of each other)

20

CPS-ASEC Reported Earnings

Matched DERS Earnings

$1 to $9999

$10,000 to

$19,999

$20,000 to

$29,999

$30,000 to

$39,999

$40,000 to

$49,999

$50,000 to

$59,999

$60,000 to

$69,999

$70,000 to

$79,999

$80,000 to

$89,999

$90,000 to

$99,999$100,000 or more

Sum of Diagonals

$1 to $999 Same -1 -2 -3 -4 -5 -6 -7 -8 -9 -10

$10,000 to $19,999 +1 Same -1 -2 -3 -4 -5 -6 -7 -8 -9 Sum

-10

$20,000 to $29,999 +2 +1 Same -1 -2 -3 -4 -5 -6 -7 -8 Sum

-9

$30,000 to $39,999 +3 +2 +1 Same -1 -2 -3 -4 -5 -6 -7 Sum

-8

$40,000 to $49,999 +4 +3 +2 +1 Same -1 -2 -3 -4 -5 -6 Sum

-7

$50,000 to $59,999 +5 +4 +3 +2 +1 Same -1 -2 -3 -4 -5 Sum

-6

$60,000 to $69,999 +6 +5 +4 +3 +2 +1 Same -1 -2 -3 -4 Sum

-5

$70,000 to $79,999 +7 +6 +5 +4 +3 +2 +1 Same -1 -2 -3 Sum

-4

$80,000 to $89,999 +8 +7 +6 +5 +4 +3 +2 +1 Same -1 -2 Sum

-3

$90,000 to $99,999 +9 +8 +7 +6 +5 +4 +3 +2 +1 Same -1 Sum

-2

$100,000 or more +10 +9 +8 +7 +6 +5 +4 +3 +2 +1 Same Sum

-1

Sum of Diagonals

Sum+10

Sum+9

Sum +8

Sum +7

Sum+6

Sum+5

Sum+4

Sum+3

Sum+2

Sum+1

SumSame

21

The Population with Earnings who Matched

• 52% of all persons 15+ with earnings are within $10,000 of each other—79% are within plus or minus $20,000 of each other• Reporters (no imputes) do best with 61%

of those with earnings within $10,000 of each other—88% within plus or minus $20,000

22

The Population with Earnings who Matched (continued)

• More persons with item imputes were within $10,000 of each other, 28%, than those with whole imputes, 24%

• 73% and 72% respectively were within plus or minus $20,000

23

Distribution of ASEC and DER Earning Class Differences in 2005(All)

-9 -8 -7 -6 -5 -4 -3 -2 -1

Sam

e +1 +2 +3 +4 +5 +6 +7 +8 +9 +100%

10%

20%

30%

40%

50%

60%

70%

1% 1% 2% 4%

11%

52%

16%

6%3% 1% 1% 1%

$10,000 Class Categories

ASEC < DER ASEC > DER

24

Distribution of ASEC and DER Earning Class Differences in 2005(No Imputes)

-10 -9 -8 -7 -6 -5 -4 -3 -2 -1

Sam

e +1 +2 +3 +4 +5 +6 +7 +8 +9 +100%

10%

20%

30%

40%

50%

60%

70%

1% 2%

10%

61%

17%

4% 2% 1% 1%

$10,000 Class Categories

ASEC < DER ASEC > DER

25

Distribution of ASEC and DER Earning Class Differences in 2005 (Item Imputes)

-10 -9 -8 -7 -6 -5 -4 -3 -2 -1

Sam

e +1 +2 +3 +4 +5 +6 +7 +8 +9 +100%

10%

20%

30%

40%

50%

60%

70%

1% 1% 2% 3% 4%7%

13%

28%

16%

9%5% 3% 2% 1% 1% 1% 1%

$10,000 Class Categories

ASEC < DER ASEC > DER

26

Distribution of ASEC and DER Earning Class Differences in 2005 (Whole Imputes)

-10 -9 -8 -7 -6 -5 -4 -3 -2 -1

Sam

e +1 +2 +3 +4 +5 +6 +7 +8 +9 +100%

10%

20%

30%

40%

50%

60%

70%

1% 2% 3% 5%8%

15%

24%

15%10%

6%3% 2% 1% 1% 1% 1%

ASEC < DER ASEC > DER

27

Substituting Wages for Earnings

• Have a slight increase in the number of persons who matched which rose from 52% to 56 % overall reporting in the same category

• Since wages accounted for 93% of earnings, increase is small

• Larger increase was for reporters (no imputes) which rose from 62% to 71% reporting in the same category

• Little change for imputed

28

Distribution of ASEC and DER Wages Class Differences, 2005: (All)

-10 -9 -8 -7 -6 -5 -4 -3 -2 -1 Same +1 +2 +3 +4 +5 +6 +7 +8 +9 +100%

10%

20%

30%

40%

50%

60%

70%

1% 1% 2% 3%

11%

56%

16%

5%2% 1% 1%

$10,000 Class Categories

CPS-ASEC < DER CPS-ASEC > DER

Percent of Total

29

Another View

• Looked at the ratio of mean earnings (ASEC/DER)

• Aggregate estimates are very similar---ASEC slightly higher--

• Thus ASEC aggregates obtained will be slightly higher

30

Ratio of Mean Earnings, ASEC/DERPercent

(Unit Counts)

ASEC/DER Total Not imputed Imputed

ASEC earnings < 5% different from DER 99.9% 99.9% 99.9%

5% to < 10% different 98.9% 98.9% 99.4%

10% to < 15% different 97.1% 97.0% 97.6%

15% to < 20%different 95.4% 95.4% 95.5%

20% to < 25% different 92.8% 92.8% 92.5%

25% < 50% different 86.3% 89.3% 82.1%

50% < 75% to < 100% different 71.7% 87.9% 62.6%

75% to , 100% different 73.0% 104.1% 54.1%

100% or more different 370.0% 357.9% 379.4%

Total 103.0% 102.6% 104.3%

31

Ratio of ASEC to DER Earnings by Demographic Group, Unit Counts

ASEC/DER Total Not imputed ImputedAll 103.0% 102.6% 104.3%GenderMen 104.1% 102.8% 104.3%Women 102.7% 102.2% 104.4%RaceWhite 102.9% 102.7% 103.4%Black 104.0% 102.3% 110.1%Hispanic 102.8% 100.0% 111.6%Other 102.6% 101.0% 106.8%Age18 to 44 104.8% 103.7% 108.6%44 to 64 100.4% 100.9% 99.2%65 and Over 110.7% 107.9% 116.6%Family TypeSingle Parent 107.9% 104.9% 118.7%Married with Kids 101.9% 101.9% 102.0%Married without Kids 102.3% 101.9% 103.3%Unrelated Individual 105.7% 104.8% 108.9%

32

The Poverty Population

• Have results for the poverty population—both total and for policy important demographic groups

• Since many eligibility limits hover around poverty, these results provide important information

33

The Poverty Population

• When DER earnings substituted for ASEC earnings, majority of persons do not change poverty status

• Results hold for demographic groups as well • Divergence is roughly symmetrical—slightly

more persons entered rather than leave poverty

34

Change in Poverty Status When DER Earnings Substituted for ASEC Earnings: 2005

No Imputes Item Imputes Whole Imputes Total

No Change in Poverty Status 94.4% 92.8% 89.2% 93.7%

Change from Poor to Non-Poor 2.8% 3.0% 5.3% 3.0%

Change from Non-Poor to Poor 2.9% 4.3% 3.0% 3.3%

Net Change Non-Poor to Poor 0.1% 1.3% 0.3% 0.2%

35

Implications for Policy

• The changes do not dramatically alter our understanding of the size and composition of the poor as measured by the Federal government• Still need to analyze persons who did not

match to DER (JSM 2012 and matches with highly dissimilar incomes)• Supplemental Poverty Measures