Review of 2016 MARCO Survey

Presented at Waikato Regional Council, Hamilton27 November 2015

Judy Oakden Pragmatica Limited – a member of the Kinnect Group

www.kinnect.co.nz or www.pragmatica.co.nz

Tel: +64 4 479 8070 Mob: +64 27 479 8070

What are we reviewing?01

What we will cover today

02 What have we found?

03 What’s next?

What are we

reviewing?

4

What are we reviewing?

• Reviewing the purpose and objectives of the MARCO survey in order to understand the value and relevance to local authority reporting processes, including through the Waikato Progress Indicators.

• Assessing the existing questions and providing recommendations for deletion of any existing questions, and/or the addition of new questions as noted in the Issues and Options paper.

• Considering the audience and use - particularly local authorities within the Waikato region.

• Responding to recommendations in the Issues and Options paper.

Key findingsWhat have we found?

6

Sample size/methodology issues

We found the issues are greater than just sample size and include:

•There are wording changes in both the General Social Survey (overall life satisfaction question) and the Quality of Life Survey.

•Further, the interviewing methodology for the Quality of Life Survey has changed from a telephone interview to a mix of self completion and online survey responses to achieve a better representation and response rate.

The Issues and Options paper recommended “a minimal sample size (sampling frame) of n = 1500 to enable TA level disaggregation (plus option to boost local sample sizes)”, with which we agree.

We recommend this is an appropriate juncture to review the whole survey methodology, not just the questions asked and the sample sizes.

.

7

Are we after cost saving or cost efficient surveying?

The MARCO survey needs to not only provide data for the WPI but can also provide additional value to the region if it:

•Provides information on areas of importance to the region, which are also areas for which the Regional Council has responsibility.

•Is sufficiently robust to inform policy decisions in the Waikato Region.

•Is comparable with other regions where the Waikato wants to make competitive claims.

The Issues and Options paper said “For cost efficiency and to maximise value to stakeholders, the issues and options paper recommended adopting a core set of questions for the 2016 MARCO Survey based on WPI need, and consult around possible additional questions”.

We agree with this view, but have a different view as to how this might be achieved. There is generally a break even point for surveys of around 10 minutes of questioning. Further, use of pre-coded questions can capture important data cost-effectively.

8

Currently mixed value and relevance of the MARCO survey to local authority reporting processes, including through the Waikato Progress Indicators

Local Authority reporting processes:•Need to have sufficient sample size to be relevant to local TA’s. Maybe allow TA’s to buy booster samples?•Some of the MARCO questions are included in TA surveys and the difference in results can be challenging for TA’s to explain. This needs to be thought through.

Data collection for Waikato Progress Indicators recommended in the Issues and Options paper included:•Community pride •Physical activity •Cultural respect •Community engagement  •Perceptions of safety •Life satisfaction.  

We suggest MARCO consider reviewing the new questions from other surveys before making a final decision about the way questions are asked. The issue is a trade off between time series data or more up to date questioning that is comparable with other surveys.

9

Possible new areas for questioning

From Issues and Options paper:

•“Recycling – A suggested measure should be identified from existing instruments for this purpose.•Participation in a sports club, church or spiritual group, online networks, people from work or school, etc. Options may be identified through forthcoming harmonisation mapping”.

However, these areas of questioning did not come up as strongly with stakeholders as some other areas.

From discussions with stakeholders

•Housing access and affordability.•Issues linked to access of services in rural areas.•Transport options (particularly to aid policy development around cycle ways).•Ways to demonstrate the competitive benefits of the Waikato region in telling the Waikato Story.•Carefully reviewing Quality of Life survey for other possibly new useful wellbeing and access questions. •Using pre-coded lists for some of the existing open ended questions rather than losing them altogether.

Moving forwardWhat’s next?

11

Reaction – guidance for reporting back to MARCO

• In general …..

• Except that…..

• On the one hand….. But on the other hand…..

• What surprises me is that….. What is missing is that …..

• What still puzzles me is that …

What are your reactions to the findings today?

What do we do next?

contact information

Judy OakdenPragmatica Limited – member of the Kinnect Groupjudyoakden@gmail.com

www.pragmatica.nzwww.kinnect.co.nz

Thank you