ONS’s Data Collection Programme: What does it mean for ...€¦ · Non-Survey Data • Non-survey...

ONS’s Data Collection Programme:What does it mean for Labour Market data?

Ian O’ Sullivan, Ed Dunn & Andrew Phelps

Office for National StatisticsSocial Survey Division

Context

• Better Statistics, Better Decisions – a strategy for UK statistics

• Spending Review• New analytical needs; new data sources• We have to deliver the savings; we have to

remain relevant and useful• So we have to change the way we do things

Data Collection Transformation

As part of our Spending Review 15 bid, we madethe case to invest in Data Collection Transformation:

“To rebalance ONS’ data collection activity significantly toward wider, more integrated use of administrative data sources, thereby reducing our reliance on large population and business surveys….Further, the remaining survey operations will become more efficient through a move from paper and personal interview-based collection to online data collection .”

Benefits

• Higher quality statisticsMore relevant, more detailed, more timely, more

accurate, more coherent• At reduced cost• With reduced respondent burden• With reduced staff stretch• More able to keep up to date with rapidly changing

technology• With potential to wider Govt DC services

hence wider govt savings and /or income for ONS• Reduced delivery risk ( in the longer term …)

Non-Survey Data

• Non-survey data is not a new idea!• Non-survey data is already used in official

statistics• ONS has over 500 non-survey data sources

- Bank of England, DWP, HM Treasury provide financial data for National Accounts

- Department for Business and Innovation* provides data on learner records and trade union membership for migration and GDP estimates

Digital Economy Act (2017)

• The Act clears the numerous legal barriers that have evolved over time preventing ONS getting administrative data, when we need it.

• It has replaced about one-quarter of the 2007 Statistics and Registration Service Act.

• Gives ONS legal rights to access data held by Government departments, other public bodies, charities and large/medium-sized businesses where that data is needed to support our work.

Digital Economy Act (2017)

• Gives ONS an important role overseeing a new framework to support the wider UK research community, within government and beyond.

• Enables accreditation of researchers who want to access (and link together) administrative datasets for research projects for the public good.

• This builds on the success of the ONS Approved Researcher programme and newer initiatives such as the Administrative Data Research Network.

Digital Economy Act (2017)

• Working on new codes of practice that underpin the statistics and research strands in the Act.

• ONS data to start becoming available from March 2018.

How will ONS use non -survey data?

• Improve survey coverage• Replace survey questions• Targeted sampling• Improved sample frames• Improved quality• New outputs• More timely data • Reduced burden on respondents

Where is our focus?

• Enhanced sample frame

• Question replacement

Enhanced Sample Frame

• Some addresses on the sample frame contain ineligible households such as:

- Business addresses- Vacant properties- Derelict properties- Second homes

• Supplement sample frame with admin data (LA Council Tax) to identify these types of properties

• AddressBase likely to be sample frame

Identifying types of residences/ communal establishments

Building owners billed for council tax:• House in multiple occupation, e.g. bedsits

with communal washing and cooking facilities • Residential and nursing homes• Some hostels• Convents & some vicarages• Properties occupied by asylum seekers• Student halls of residences

Question replacement LFS

• Employed/Self employed

• Left last job• Started job• Redundancy• No. of employees• NI Payments• Use of computer (at

home)

HMRC P45/P60 Redundancy

Payments Service

HMRC Full Payment Summary HMRC

Expenses and Benefits

HMRC NI Database

Earnings

• Gross pay• Pay period• Net pay• Usual pay• Statutory sick pay• Main job and second

job

• New outputs on shared parental leave?

HMRC PAYE/SA

HMRC Full Payment Summary

Statutory Sick Pay

Statutory Maternity/Paternit

y

Statutory Adoption Pay/ Shared

Parental Leave

Respondent Characteristics

• Country of birth• Marital status• Living as a couple• Children in

household• Accommodation as

part of job• Status of maternity

leave

Births and Deaths

Register

Benefits Data (Child Tax

Credit/ Child Benefit)

Marriage and Civil

Partnerships

HMRC Expenses and Benefits Tax

Data

HMRC Statutory Maternity Payments

Statistical Considerations

• Concept alignment between the non-survey data and survey data

• Coherence between survey and non-survey estimates

• Quality assessment e.g. timeliness, accuracy, vacuity

• Knowledge of how data supplier collects and processes data before delivery to ONS

• Engage with and inform stakeholders• Methodologies to link the data

Transformation of the LFS

• Transformation not translation

• Questionnaire will need to be�shorter�simpler�designed for self-completion and �multiple modes

• Respond to the Bean Review, and user needs (respondents & stakeholders)

Research and testing

Approach to research planning:• Focus on research to support prototype labour market

quant test• Main focus is online although other modes are

considered and aim to have questions that are equivalent

• Initial research scope: - review of existing LFS questionnaire content- development and quali testing of a Labour Market

survey – online, face-to-face, and telephone- respondent engagement strategy and materials

Testing plan

• Online take up tests (July and Sep 2017)• Online retention test (Nov 2017)• Statistical mixed mode test (Spring/ Summer 2018) – 1 wave• Longitudinal test (2019/20)• Parallel run start (2019/20)

• Fresh samples for each test• Iterative process – detail of later tests likely to change following results of early tests• Full analysis of outcomes, questionnaire data and relevant paradata

Initial 2017 online take up test (Test 1)

Purpose• Provide an early indication of uptake for a Labour Market Survey online• Provide evidence of the most effective comms strategy to maximise initial online uptake (excluding incentive considerations)• Test take-up of enquiry line service and nature of calls

• No specific test of q’re content/look or analysis of survey data – but some evidence of q’re length/proxy rates/partial completion.

Initial 2017 online take up test (Test 1)

Basic design• Mail out to c.37,000 households• Test for most effective:

- Number of comms (prenote+invite+reminder vs. invite+2 reminders vs. invite+1 reminder)- Envelope colour (brown vs. white)- Day of week the invite letter is received (Fri vs. Mon)- National branding on envelopes (Wales and Scotland only)

• Issue to 12,600 addresses in each of England, Wales and Scotland • ‘New’ sample of addresses•Test fieldwork 12th-24th May 2017 7th-19th July 2017

Overall response

• Number of visits to ONS website page: 8,891 • Number of clicks on ‘Start now button’: 8,280 • Entered access code + started survey: 6,835 (19.9%)

• Of these 6,835:• Full household response – 5,906 (17.2%)• Partial household response (usable) – 643 (1.9%)• Partial household response (unusable) – 286 (0.8%)

Communications strategy

• Three test groups:

• Invitation letter + 1 reminder - 16.3%• Prenote letter + invitation letter + reminder - 18.4%• Invitation letter + 2 reminders - 19.5%

Mailing day of the week

• Wednesday - 18.8%• Friday - 17.4%

Envelope colour

• Brown - 18.4%• White – 17.7%

Nation specific envelope branding

Country With branding (%)

Without branding (%)

Total (%)

England - - 19.6

Scotland 16.8 17.2 17.0

Wales 18.0 17.4 17.7

Further 2017 online test (Test 2)

Purpose• Provide another indication of uptake for a Labour Market Survey online • Establish the likely most cost effective incentive strategy• No specific test of q’re content/look or analysis of survey data - but further evidence of q’re length/proxy rates/partial completion with a slightly longer q’re

Further 2017 online test (Test 2)

Basic design• Mail out to c.40,000 households• Test impact of:

No incentive vs.£5 unconditional + £10 conditional vs.£5 unconditional vs.Non-monetary incentive (tote bag)

• ‘New’ sample of addresses• Online test fieldwork September 2017

September 2017 online test (Test 2)

Basic design• Mail out to c.40,000 households• Test impact of:

No incentive vs. (22.5%)£5 unconditional + £10 conditional vs. (30.8%)£5 unconditional vs. (29.3%)Non-monetary incentive (tote bag) (27.0%)

What next? – November 2017

• November 2017 – online follow up of Test 1 respondents (test 1b)

-Test 1 – 6,327 households responded, 5,226 gave an email or phone number (83%)- Test 1b will follow-up online the 5,226 with the Test 2 q’re- No information will be rotated forward- What proportion will do it again?- Will we get –ve feedback about not rotating forward information?

What next? – Spring/Summer 2018

•Spring/Summer 2018 – 1 wave mixed mode test:- What happens when the interviewers visit online non-responders? What response rate do we get?- What do outputs look like from mixed mode data?- Issue c.11-12k online, c. 8k to field.- Use NISRA to host online in Blaise- Scope to do an online Wave 2

-3 months later and/or 12 months later

Prototype Labour Market Survey

• Need to review entire LFS content and consider non-survey sources

• 600 Questions & DVs - Labour Market and Non-Labour Market

• Labour Market core questions driven by the LM Framework

• Non-labour market to be driven by user requirements

• Content will change dependent upon findings of work on value of non-survey data

Prototype Core Labour Market Survey

Seven key LM variables:• CURED8 ‘Current Education Received’• DURUN ‘Duration of ILO Unemployment’• FTPT ‘Whether working full-time or part-time’• INECA05 ‘Economic Activity’• REDUND ‘Whether made redundant in the

last 3 months’• SECJMBR ‘Whether second job/status in

second job’• SUMHRS ‘Total actual hours worked in main

and second job’Questions required to derive these seven variables

are prioritised.

Key Labour Market Variables

Non-labour Market Data

• Hundreds of variables on current LFS• What can non-survey data do?• Improve?• Replace?• New analysis?• ONS will investigate the efficacy of

administrative sources• Survey collection

Any questions?

Contacts:ian.o’sullivan@ons.gsi.gov.uk

ed.dunn@ons.gsi.gov.ukandrew.phelps@ons.gsi.gov.uk

Andssd.transformation@ons.gsi.gov.uk