Evaluation of the Primary Mental Health Initiatives ... · and Economics Research Unit (AERU), Lincoln University • utilisation of data from HealthStat (section 8.6): Barry Gribben

–

Evaluation of the Primary M

ental Health Initiatives:Sum

mary Report

–

2009

Evaluation of the Primary Mental Health Initiatives: Summary Report

Evaluation of the Primary Mental Health Initiatives Summary Report

Citation: Dowell AC, Garrett S, Collings S, McBain L, McKinlay E, Stanley J. 2009. Evaluation of the Primary Mental Health Initiatives: Summary report 2008.

Wellington: University of Otago and Ministry of Health.

Published in July 2009 by the Ministry of Health

PO Box 5013, Wellington, New Zealand

ISBN 978-0-478-31907-1 (print) ISBN 978-0-478-31908-8 (online)

HP 4754

This document is available on the Ministry of Health’s website: http://www.moh.govt.nz

http://www.moh.govt.nz

Acknowledgements and Contributors The evaluation team would like to thank the initiative teams for all their hard work and support in providing evaluation data. We also appreciated the warm reception we received when visiting the initiatives, and the invaluable discussions about various aspects of these projects. We thank Ray Prebble for help with editing the report.

Core evaluation team The core evaluation team comprises the following members: • Professor Tony Dowell, Principal Investigator • Sue Garrett, Project Manager (from October 2005) • Dr Sunny Collings • Dr Lynn McBain • Eileen McKinlay • Dr James Stanley (from July 2007).

Contributors In conjunction with the core evaluation team, chapter contributors undertook significant data collection, assisted with writing a chapter and/or acted in an advisory capacity for a chapter, as follows:

• service user data: Suzanne Pitama (who also advised on various aspects of evaluation related to Māori),Valerie Bos, Nita Hill, Dr Lynn Pere, Jayden MacRae and John Grant

• outcome measures: Jayden MacRae, John Grant, Sina Barrott, and South East and City PHO (SECPHO)

• economic analysis: Des O’Dea, Peter Hayes and Professor Caroline Saunders, Agribusiness and Economics Research Unit (AERU), Lincoln University

• utilisation of data from HealthStat (section 8.6): Barry Gribben and CBG Research.

Evaluation of the Primary Mental Health Initiatives: Summary Report iii

Contents

Key Points ix

Executive Summary xii

Chapter 1: Introduction 1

1.1 Primary mental health care 1

1.2 The role of primary health care in mental health care 3

1.3 The development of Primary Mental Health Initiatives 3

1.4 Issues from existing literature 4

1.5 The evaluation of the Primary Mental Health Initiatives 6

Chapter 2: Methods 9

2.1 Qualitative methods 9

2.2 Quantitative methods 11

2.3 Talking therapy survey 13

2.4 Ethics approval 13

Chapter 3: History 17

Chapter 4: Description of Initiatives: Models of Care 18

4.1 Introduction to the initiatives 18

4.2 Models of care 22

4.3 Key points 30

Chapter 5: Case Study Synthesis 31

5.1 The case studies 31

5.2 Factors optimising implementation of the initiatives 32

Chapter 6: Workforce 34

6.1 Workforce description 34

6.2 Synthesis of commentary on the different roles 37

6.3 Implications for the development of a primary mental health care workforce 38

6.4 Key points 40

Chapter 7: Talking Therapies in Primary Mental Health Care in New Zealand 42

7.1 Key results of the therapist survey 42

7.2 Kaupapa Māori-informed models 44

7.3 Other results of the therapist survey 45

7.4 Discussion 45

7.5 Key points 46

Evaluation of the Primary Mental Health Initiatives: Summary Report v

Chapter 8: Service-user Data 47

8.1 Key points 47

8.2 Demographics 48

8.3 Diagnoses 49

8.4 Interventions 52

8.5 Service-user perspective 53

Part one: Service users identifying as Māori 55

Part two: All other service users 55 8.6 Utilisation of data from HealthStat (authored by CBG Research) 56

Chapter 9: Outcome Measures 58

9.1 Methods and analysis 58

9.2 Overall scores 59

9.3 Individual variation 61

9.4 Outcome scores by diagnostic group 64

9.5 Ethnicity 65

9.6 Six-month follow-up 66

Chapter 10: Economic Analysis 68

10.1 Analysis of positions created and services provided by initiatives 68

10.2 Findings on cost effectiveness 69

10.3 Key points 71

Chapter 11: Conclusions and Service Implications 72

11.1 Context 72

11.2 The overall effectiveness of the PMHIs 72

11.3 Appropriateness of service provision 74

11.4 Models of care 75

11.5 Workforce 76

11.6 Information technology, clinical decision support and assessment tools 77

11.7 Limitations 78

11.8 Further research 78

References 79

Evaluation of the Primary Mental Health Initiatives: Summary Report vi

List of Tables Table E1: Proportions of service users showing different magnitudes of improvement xix

Table 2.1: Summary of interviews completed 14

Table 4.1: Primary Mental Health Initiatives, by DHB region 18

Table 4.2: Types of service available through each initiative 20

Table 6.1: Common positions created and their key components 34

Table 6.2: Training needs identified by new primary mental health clinicians 39

Table 7.1: Counts and percentages of respondents using specific therapeutic models 43

Table 7.2: Counts and percentages of respondents using kaupapa Māori models 44

Table 8.1: Major diagnostic categories and sub-classifications 50

Table 8.2: Hierarchical diagnostic category, by ethnicity 51

Table 8.3: Intervention visit types 52

Table 9.1: Proportions of service users showing different magnitudes of improvement 63

Table 9.2: Number of episodes with pre-intervention scores, by main diagnostic group 64

Table 10.1: Range of costs for different services offered by initiatives 68

Table 10.2: Set-up costs for initiatives, including breakdown by IT and other costs 69

Table 10.3: Costing information and coverage of service users with both pre- and post-intervention scores 69

List of Figures Figure E1: Key structural elements of a PMHI xv

Figure E2: Service-user pathway (in the context of the structure of Figure E1) xvi Figure E3: Distribution of improvement scores (converted to equivalent points on the K10) across

4263 service users with paired pre- and post-intervention scores xviii Figure 2.1: Alignment of aims and methods in a mixed methods evaluation of PMHIs 14

Figure 5.1: The eight case studies on a continuum, based on the type of input

Figure 9.1: Mean outcome scores (as percentage points) for service users at pre- and post-

32

Figure 8.1: Age and gender of service users compared with PHO enrolled population 48

Figure 8.2: Ethnicity of initiative service users compared with PHO enrolled population 49

intervention 60

Figure 9.2: Random effects meta-analysis of improvement across all initiatives (presented in random order) 61

Figure 9.3: Distribution of pre-intervention scores (as percentage points) across all service users 62

Figure 9.4: Distribution of improvement in scores (as percentage points) across all service users with paired pre- and post-intervention scores 63

Figure 9.5: Average outcome scores (percentage points) at pre-intervention and post-intervention, by exclusive diagnostic category 64

Figure 9.6: Pre-intervention and post-intervention scores (as a percentage of the scale maximum), by ethnicity 66

Figure 9.7: Mean outcome measure scores (as percentage points) at pre-intervention, post-intervention, and six months after cessation of treatment 67

Evaluation of the Primary Mental Health Initiatives: Summary Report vii

Key Points Overview • Clinically significant mental health problems are common in New Zealand. Many people,

particularly those with mild to moderate conditions, are first seen in primary health care and general practice settings.

• The Primary Mental Health Initiatives (PMHIs) represent a significant investment in primary care service delivery and infrastructure. The initiatives were delivered within a timeframe requiring a rapid evolution of service development, and were sustained over a two-year period.

• A range of service delivery models was developed, offering choice to both service users and practitioners. All services were offered at no charge to services users. Every model contained some elements specific to local need.

• The PMHIs were perceived as being efficient and responsive to consumer need, and were judged a success by both practitioners and service users.

• It is unlikely that any single model could be universally applied as none contained all successful elements of an optimal model.

• No service delivery model offered an inherently superior value for money, or an inherently more cost-effective service compared to others.

• The PMHIs provided services to address the needs of service users with a wide range of symptoms and problems. The ability to address undifferentiated and sub-threshold symptom complexes, as well as well-defined conditions such as depression and anxiety, was welcomed by service users.

• Up to 80% of service users benefited from the variety of interventions offered.

• Although common, mental health disorders such as anxiety and depression are complex and it is not surprising that 20% of service users did not improve. Spontaneous remission (getting better without any treatment) is also relatively common. Several studies suggest that it would be reasonable to expect anywhere between 30 and 50% of patients to improve under ‘care as usual’ conditions and fewer under ‘no treatment’ conditions. The 80% improvement rate therefore represents a significant and beneficial treatment effect, which was generally sustained at six months in those for whom data were available.

• Service users expressed satisfaction with the care given by the initiatives.

• Mental health needs arising from mild to moderate common mental health conditions, including those involving social complexity, can be addressed by primary care.

Did PMHIs reach different population groups? • Targeting of services to high-needs populations was a challenge for many PMHIs. Well-

defined criteria for determining clinical eligibility for care are important.

• Services for Māori included both kaupapa Māori and mainstream service options, and having both options available was perceived as optimal by Māori.

Evaluation of the Primary Mental Health Initiatives: Summary Report ix

• The initiatives provided access to Māori in excess to their proportion in the enrolled population. However, given the higher prevalence of some common mental health conditions among Māori, it is likely there was still some under-utilisation of services by Māori.

• There was under-utilisation by Pacific peoples and significant under-utilisation by Asian people in the PMHIs.

• The mental health needs of children and young people overall were not sufficiently met by the PMHIs, and over half did not offer services to this group.

• Few PMHIs offered services to service users over 65 years of age.

Workforce • The success of the PMHIs was based in part on the investment made in dedicated staff

positions such as mental health co-ordinators and mental health nurses. These staff require appropriate administrative support, and need a framework for professional development and supervision.

• The PMHIs are an example of effective interdisciplinary teamwork. In the majority of initiatives, care was provided by more than one discipline. The involvement of practice nurses was not always facilitated.

• There is currently no optimally structured training or education programme for primary mental health care that includes all disciplines.

Infrastructure and methodologies • The majority of initiatives made relatively little use of advanced information technology (IT)

platforms for administrative and clinical management, or for structured clinical decision support.

• The use of clinical outcome measures was encouraged, both to support the evaluation and as a clinical tool. Further use of assessment and diagnostic tools would mean a significant change in primary mental health care practice.

• A wide range of psychological counselling and therapy approaches was used. There is a need to understand which elements of these interventions contribute to successful management.

Further work • Further work is required to define an optimal balance between psychological and

pharmacological therapies within primary mental health care.

• Further work is also required to define appropriate and acceptable funding systems to resolve issues of health needs, equity and ability to pay.

• Future development in primary mental health should clarify the appropriate contribution, in terms of funding and co-ordination, from the Ministry of Health and District Health Boards, and should attempt to achieve a degree of equity and consistency in overall service provision.1

The term ‘mental health conditions’ has been used in this report to signify the inclusion of problems with a strong social component, rather than the term ‘mental disorders’ often found in the literature. In some instances ‘mental disorder’ will still be used in the context of other literature.

Evaluation of the Primary Mental Health Initiatives: Summary Report x

1

Executive Summary There is a high prevalence of mental health conditions in the New Zealand population, with the New Zealand Mental Health Survey predicting that 46.6% of the population will meet the criteria for mental disorder sometime in their lives. The Mental Health in General Practice (MaGPIe) study found that 36% of people attending general practice had one or more of the three most commonly presenting mental health disorders: anxiety, depression or substance-use disorder. Nearly all of these people, have mild to moderate conditions, and are first seen in primary health care and general practice settings.

This report describes the evaluation of the Ministry of Health-funded Primary Mental Health Initiatives (PMHIs), which was conducted between June 2005 and November 2007. Until the introduction of the PMHIs there had been no central funding to specifically support primary mental health care, with this kind of activity limited to a small number of primary health organisation (PHO) or District Health Board (DHB) projects. As a result, the Ministry of Health recognised that particular attention should be given to the development of primary mental health services in New Zealand.

Background and methods In April 2004 requests for proposals for primary mental health care were sent out to PHOs. Assessment of the proposals was completed by November 2004, and funding was provided for initiatives proposed by 42 PHOs and distributed to 26 different groupings. Concurrently a request for proposals for an evaluation of these initiatives was advertised and awarded in June 2005 to the Department of Primary Health Care and General Practice, University of Otago, Wellington.

The Ministry of Health chose not to adopt an experimental framework or methods in the design and contracting of the evaluation of the PMHIs. Because of this the most appropriate approach to the evaluation was a mixed-method framework. The evaluation involved collecting both qualitative and quantitative data from stakeholders, clinicians and service users. Case study methods were used to collect more in-depth information on a small number of initiatives.

Service users Service-user quantitative data was obtained from 25 of the 26 initiative groupings (one initiative was not contracted to supply service-user data). No specific data format was prescribed by the Ministry of Health, and the resulting variety of data formats led to limitations in the possible analysis and interpretation of the data.

Up to 30 November 2007 16,007 service users had been seen across the 25 initiatives that were required to return information on service users. More females than males were seen, across all age categories. In terms of ethnicity, 69.3% were European, 17.5% were Māori and 4.3% Pacific peoples.

Evaluation of the Primary Mental Health Initiatives: Summary Report xi

The majority of service users (69%) referred into the initiatives through primary care were given a diagnosis of depression. A large number of different interventions were identified, resulting in 53,001 visits to providers. General practice-based interventions were the most common intervention received. These were predominantly GP consultations funded beyond the standard 15 minutes, usually to either 30 or 45 minutes, and 49.7% of all service users made at least one visit for this kind of intervention.

There was a wide range of clinical presentation to the PMHIs, and the service delivery challenges differed from those arising in secondary care models of community-based mental health care. Service users had a complex range of issues motivating them to seek help, including a past history of psychological distress, physical health problems, unresolved post-traumatic stress disorders, grief, abuse, social issues and ongoing depression/anxiety. The assessment process developed in many initiatives was thus both comprehensive and holistic, covering psychological, physical, gender and social/family issues.

In some cases service users required quite substantial support because of the multiple stresses they experienced. An appropriate match between psychological therapist and service user was perceived as important, this being reported particularly by Māori and Pacific peoples in the qualitative data. The ‘no cost’ nature of the service was also mentioned as an important factor by service users.

Models A variety of different models of care have evolved, based on meeting local need and dependent on local PHO capacity; for example, some initiatives employed mental health staff while others contracted various services out. As a result of this local adaptation, models that evolved in large organisations would be unlikely to work effectively in smaller initiatives. In particular, Māori and Pacific services have tailored their services and workforce to their local populations and context.

Increased access to community and social support networks and therapists was enabled by the use of mental health co-ordinators in some of the models. It was important that the service models were able to address the (frequently present) complex social needs of service users. Appropriate liaison and integration with secondary care services was also an important part of effective intervention. The key factors enabling successful liaison were communication and an understanding of the respective roles of the primary and secondary services.

The evaluation used eight case studies to examine aspects of the initiatives in greater detail. Successful service delivery was linked to having adequate initial preparation time, establishing good relationships with referring practitioners, and developing a definition of mental illness and mental health problems that included life complexity. Initiatives and referrers acknowledged the aims of the request for proposal, and high-needs populations were generally targeted, although the criteria for determining ‘high needs’ differed.

xii Evaluation of the Primary Mental Health Initiatives: Summary Report

The evolution of different models highlighted the importance of balancing work undertaken by the existing primary care team of general practitioner (GP) and practice nurse (PN) with that done by the new primary mental health clinicians, such as the co-ordinator/nurse role, and/or referral to external providers. In a number of the case studies practice staff preferred to have a primary mental health clinician available to undertake more intensive work, due to lack of time and expertise or knowledge of community resources.

The complexity and effectiveness of teamwork varied across case studies. Initiatives that were co-located in general practices appeared to have a positive influence on teamwork, both within practices and between general practice staff and initiative staff.

An optimal Primary Mental Health Initiative No single initiative incorporated all the features the evaluation team considered to be core aspects of an optimal model. By combining positive features of all the different models, a conceptual schema has been devised for an effective ‘generic’ initiative, although we recognise that appropriate local adaptation will depend on both resourcing and community need. This schema is shown in Figures E1 and E2.

Features of an optimal model include support from the DHB, an effective IT platform, and incorporating training, health promotion and liaison with both secondary care and other sectors into the model. Over time, primary care practitioners and therapists need to develop a new skill mix tailored to the needs of initiatives. These skills include: • assessment • use of outcome tools • brief interventions and talking therapies • motivational interviewing • self-management • medication use.

An effective patient care pathway would depend on identifying appropriate roles for existing and new providers, the adoption of stepped care, and appropriate regard for service-user choice within the available resources.

Evaluation of the Primary Mental Health Initiatives: Summary Report xiii

Optimal model for a Primary Mental Health Initiative (PMHI)

Figure E1: Key structural elements of a PMHI

xiv Evaluation of the Primary Mental Health Initiatives: Summary Report

Figure E2: Service-user pathway (in the context of the structure of Figure E1)

Meeting the needs of Māori and Pacific peoples New primary mental health services aimed to be responsive to Māori and used both kaupapa Māori and mainstream approaches. In the mainstream services some PHOs found it difficult to translate ‘by Māori for Māori’ into practice due to a lack of guidelines and difficulty accessing a Māori workforce. Although some initiatives did not have proportional rates of Māori enrolment, the overall numbers across all the initiatives showed proportionately greater access to Māori service users than other population groups when compared to numbers in the enrolled population. However, given the known prevalence of some disorders among Māori it is probable there is still some service under-utilisation.

Primary mental health services for Pacific peoples have recognised the need to include health promotion and de-stigmatisation, as some Pacific peoples and communities appear reluctant to address or acknowledge mental health issues. There also appear to be significant variations in Pacific people’s belief systems about mental health and treatment options, and programmes need to accommodate or reflect this. Further work also needs to be done to increase the Pacific mental health workforce. Overall, there was under-utilisation of services by Pacific peoples compared to their proportion in enrolled populations, and more work needs to be undertaken to understand the barriers to utilisation.

Evaluation of the Primary Mental Health Initiatives: Summary Report xv

Workforce All the PMHIs have created new roles and positions in New Zealand primary care, the most common being those of a primary mental health co-ordinator, mental health nurse or primary mental health clinician. The way work is balanced between existing primary care staff and newly created roles differs across PMHIs. Workforce imperatives include consideration of the use of existing versus newly created roles, as well as decisions regarding development, the appropriate use and maintenance of skills, exploring appropriate training options, addressing high levels of demand, and future workforce additions or development.

The professional development needs of primary mental health clinicians were varied and dependent on previous experience. Common learning needs were for a greater understanding of the nature of primary health care service delivery, and specific training in brief interventions. Existing primary care staff wanted mentoring in identification and screening, diagnosis and management, mainly delivered through a case-based learning approach.

Psychological therapies The opportunity to access brief psychological therapies from primary care (and the flexibility in some situations to provide more than brief therapies, if needed) is filling a large unmet need. The approach taken to securing these services varied according to the model of care provision and the resources available locally. The therapy workforce is extremely varied in terms of qualifications, experience and type of treatment offered, and so quality assurance processes are imperative. The majority of therapists and counsellors considered their training and experience had equipped them to work effectively in the primary care domain, and they adopted a pragmatic, eclectic approach. There was some desire for additional training, especially on types of therapy and core knowledge of mental conditions.

Service outcome measures The analysis of outcome measures from this evaluation is complex, given the lack of an experimental framework, and quantitative data should be interpreted with some caution. Different initiatives used different outcome measures and there was variability in data collection. Pre-intervention scores were not significantly different between the different models of care groupings, and the majority of service users showed improvement in their post-intervention scores. (See Table E1 and Figure E3.)

Outcomes were measured using three different scales across initiatives: the Kessler 10 (K10), the Patient Health Questionnaire (PHQ-9), and the General Health Questionnaire (GHQ-12). This necessitated transforming the raw data so that it could be combined across all the initiatives. To achieve this, scores for each individual were converted to a percentage of the maximum score on that scale (referred to as ‘percentage points’). Improvement results have been re-expressed in this executive summary in terms of scores on the K10 scale (a minimum score of 10, a maximum of 50) by conversion back from the percentage point scale, but include data collected on all three measures.

xvi Evaluation of the Primary Mental Health Initiatives: Summary Report

A meta-analysis of improvement scores showed a mean improvement in scores of 11.7 points, expressed in terms of the K10 outcome measure (95% confidence interval [CI]: 8.9 to 14.7 points). This was based on data from 4263 individuals, covering 14 of the 26 initiatives in the evaluation. There was some variability between initiatives in terms of the mean improvement in scores, but all the initiatives had a mean positive impact on patient outcome measures (this data is presented in the body of the Evaluation of the Primary Mental Health Initiatives: Extended report (Dowell et al 2008).

Figure E3: Distribution of improvement scores (converted to equivalent points on the K10) across 4263 service users with paired pre- and post-intervention scores

Mean Zero change

400

300

200

100

0

Serv

ice-

user

cou

nt

-40 -30 -20 -10 0 10 20 30 40 50

Improvement in score (K10 points equivalent)

Figure E3 shows the distribution of score improvements following intervention for the 4263 service users with paired pre- and post-intervention scores.2 Improvement expressed as points on the K10 is presented on the horizontal axis, grouped into bands of 2 percentage points: for example, the first bar to the right of 0 shows the count of service users showing an improvement of either 0 or 1 points; the bars to the left of the zero indicate the counts of service users with an increase in score following intervention (ie, symptoms of psychological distress increased). The superimposed black curve shows the normal distribution expected in this data based on the mean (represented for this data set by the vertical black line) and standard deviation.

The mean improvement noted in Figure E.1 is slightly different from that calculated under the meta-analysis framework, due to the more nuanced handling of variability between initiatives in the meta-analysis.

Evaluation of the Primary Mental Health Initiatives: Summary Report xvii

2

It is possible to calculate from this distribution of scores the proportions of service users seen whose improvement scores fall above certain thresholds, based on the actual distribution of the scores. It can be seen from Table E1 that the vast majority of service users showed at least some improvement in score following intervention, and more than half showed an improvement equating to at least 8 points on the K10 scale.

Table E1: Proportions of service users showing different magnitudes of improvement

Magnitude of improvement (equivalent no. of K10 points)

Percentage of individuals showing improvement

Greater than 0 4 points or more 8 points or more 12 points or more 20 points or more

80.9 72.4 58.1 42.3 18.0

The size of the improvements did not differ significantly between the different model-of-care groupings (see Chapter 4). Despite differences between model groups, overall common elements successfully addressed service-user need. Clinicians administering the outcome measures initially had mixed feelings about using the tools. Over time, however, clinicians gained more confidence in using the tools and recognised their benefits. Service users commented on how the tool scores validated their experience and clinical improvement.

Caution should be exercised when drawing conclusions regarding cost effectiveness by score improvement, because of the use of different outcome measures in the different initiatives, the low rate of return of outcome measures for some initiatives, and the non-standardisation of reporting of costs. The costs of services and staff (FTEs) varied greatly across regions. All services will have a certain level of set-up and ongoing running costs, but the greater the number of service users seen, the more cost effective the service becomes.

Conclusions In conclusion, the PMHIs have provided PHOs with an opportunity to develop and implement successful models of primary mental health care, which were delivered with no patient fee to generally high-needs populations. These have been well received by all involved, particularly by service users. The findings of this evaluation suggest a number of broader policy implications and changes to clinical practice.

xviii Evaluation of the Primary Mental Health Initiatives: Summary Report

Chapter 1: Introduction

1.1 Primary mental health care ‘Primary mental health care’ refers to the assessment, treatment and, when needed, ongoing management of people with mental health problems and/or addiction in the primary care setting. It encompasses promotion, prevention, early intervention and ongoing treatment for mental health.

In New Zealand, as in other OECD countries, there is a high prevalence of common mental health conditions, both in the community and among those attending general practice and primary care services. Te Rau Hinengaro (the New Zealand Mental Health Survey) predicts that 46.6% of the population will meet the criteria for a mental disorder sometime in their lives, with 20.7% having had a disorder in the past 12 months (Oakley Browne et al 2006). The Mental Health and General Practice Investigation (MaGPIe) study assessed the prevalence of common disorders presenting to general practice. Using the Composite International Diagnostic Interview (CIDI), a validated measure of diagnosis, 36% of general practice attenders were found to have one or more of the three most commonly presenting disorders: anxiety, depression or substance-use disorder (MaGPIe Research Group 2003).

Mental health conditions and their sequelae are not evenly distributed in New Zealand. The prevalence of a condition in any period is higher for Māori and Pacific peoples than for the ‘Other’ composite ethnic group, which includes Pākehā. For mental health conditions in the past 12 months, the prevalences are 29.5% for Māori, 24.4% for Pacific peoples and 19.3% for ‘Others’, which indicates that Māori and Pacific peoples have a greater burden from mental health problems.

The Primary Mental Health Initiatives (PMHIs) have been developed in an international context of increasing prevalence of these disorders (Murray and Lopez 1997) and recognition of the fact that psychological disorders may contribute more to the overall burden of long-term chronic illness than other chronic illness conditions (Moussavi et al 2007). From a primary care perspective, it is also important to reflect on the fact that mental health conditions may not fit easily or neatly into existing psychiatric classifications, and that disability may contribute more to overall primary care morbidity than a diagnostic label (MaGPIe Research Group 2005).

Current workload and work patterns within New Zealand general practice are also important drivers of the development of the PMHIs. General practice is busy, with 90% of all presented health service problems managed without referral, and 50% to 70% of mental health conditions managed solely by general practice. New Zealand has overall general practice consultation times longer than many other OECD countries, with an average of 15 minutes, but this still means that in the complex competing demands of the consultation there is a tension in responding to mental health concerns that take increased time (Klinkman 1997). In New Zealand this is compounded by the part fee-for-service system, which means that from the patient’s perspective there are financial barriers to presenting to, and continuing engagement

Evaluation of the Primary Mental Health Initiatives: Summary Report 1

with, health professionals for mental health conditions.3 By their nature these conditions are complex, taking time and repeated consultation for interventions to be successful.

Compared with the high prevalence of disorders in the general population, only a small proportion of service users actually present mental health problems to their doctor as the main reason for their consultation: four New Zealand studies have found that between 3% and 8% of service users had a mental health problem as the main presentation at the consultation.

The MaGPIe study, along with other research in New Zealand, has identified the following issues as being important for primary mental health care in the context of new initiative development and its evaluation: • the detection and recognition of ‘cases’ for treatment • access to care – barriers and facilitators • liaison and integration of care • use of ‘specialist services’.

The Primary Health Care Strategy (Minister of Health 2001) involves a new direction for primary health care, with a greater emphasis on population health and the role of the community, health promotion and preventive care, the need to involve a range of professionals, and funding based on population need rather than fees for service. There are six key priorities for primary health care: • work with local communities and enrolled populations • identify and remove health inequalities • offer access to comprehensive services to improve, maintain and restore people’s health • co-ordinate care across service areas • develop the primary health care workforce • continuously improve quality using good information.

The vehicle chosen to co-ordinate and oversee these changes was the primary health organisations (PHOs). The present 80 PHOs have developed from three main sources:

• development from existing independent practice associations (IPAs)

• third-sector or not-for-profit health organisations based around Health Care Aotearoa (HCA), some of which have strong links to Māori or Pacific providers

• a grouping of newer, often smaller, organisations, many of which are iwi-based.

This diversity leads to different philosophical approaches, capacities and rates of development, as well as different expectations with respect to infrastructure, workforce and evaluation capacity.

New Zealand has increasingly introduced a capitated payment structure since 2001, whereby general practice receives an annual payment per enrolled service user. In addition, the service user is charged a part payment at the time of each appointment. It is this part payment that acts as a barrier to service users with mental health issues, as by nature mental health issues require several appointments and therefore several part payments.


3

The change in the structure and funding of the primary health care sector brought about by the implementation of the Primary Health Care Strategy has provided an opportunity to develop primary mental health services as an integral part of PHOs. According to its philosophy, primary mental health care should be based on first-contact care that is universally accessible to people in their communities, involves community participation, and is central to the overall function of the mental health system. It should also include principles of community development and mental health promotion.

Primary mental health care was seen as an important priority for New Zealand given various measures of poor mental health status, such as the rate of suicide among young people.

1.2 The role of primary health care in mental health care Primary health care is usually the first point of contact for people with mental health and/or addiction conditions, and, as with other aspects of primary care, has a role in providing accessible and appropriate treatment and systems of referral and care for people with problems. Its ability to do so is clearly influenced by the affordability of care, as determined by government funding.

There is a strong connection between mental health and illness and other physical health conditions, especially chronic illnesses (National Health Committee 2007). A further role for primary health care is to integrate mental health and psychosocial interventions into the management of those with physical health problems, and attend to the physical health needs of those with mental health disorder. This supports the assertion that ‘there can be no health without mental health’ (Prince et al 2007).

1.3 The development of Primary Mental Health Initiatives Using the underlying principles of the Primary Health Care Strategy, the Ministry of Health secured funding in late 2003 for the Primary Mental Health Initiatives, the selection of which would be determined by a request for proposals process. The Primary Mental Health Initiatives programme in PHOs was seen as being part of a package of initiatives being undertaken to develop PHOs and support the implementation of the Primary Health Care Strategy.

Given previous funding allocation to those with enduring and more severe mental health disorders, the principal target population for these initiatives was those with a mild to moderate mental illness (estimated at 17% of the population at any one time).

The initiatives were intended to be demonstration projects, and in recognition of the limited funding available and disparities in access to care, they were targeted at known high-needs groups, with a focus on Māori, Pacific and low-income populations. The Ministry of Health drew on a variety of international literature supporting the view that primary care services have a key role in the provision of mental health services, particularly for those with a mild to moderate mental illness.


The overall aims and objectives of the initiatives outlined in the request for proposal were to:

• develop activities to reduce the prevalence and impact of mental health problems on their enrolled population, specifically education, prevention, early intervention and treatment activities

• develop the skill mix of primary health care practitioners and their ability to effectively respond to the majority of mental health problems that can be managed in primary health settings

• build effective links with other providers of mental health care, including secondary services, so that the care of those with chronic and/or long-term mental health problems is effectively co-ordinated.

1.4 Issues from existing literature The following issues are important in primary mental health care and form a basis from which to consider the PMHIs and their evaluation.

The first is the continuing debate over the effectiveness of managing common mental health conditions in general practice settings. Many studies continue to report that general practitioners (GPs) under-diagnose and under-treat mental disorders, particularly depression (Pignone et al 2002; Klinkman 2003; Hickie et al 2001). A common response to these findings has been to suggest that further training and education of GPs are required, and also that screening is an appropriate way of increasing rates of detection.

The assertion that GPs ‘miss’ many common psychological disorders, however, may well be an oversimplification of the way that doctors and service users interact in a consultation. The MaGPIe study has shown that diagnosis is linked to previous consultation rates (MaGPIe Research Group 2004), and studies suggest that there is in fact recognition of cases that would benefit from the available treatment. Given that effective screening tools are available (Arroll et al 2003) to support clinical assessment, one of the challenges for the PMHIs was to develop systems that would identify appropriate service users for assessment and intervention.

The structure of mental health care in primary care has been generally understood in terms of the ‘pathways to care’ model (Goldberg and Huxley 1980). Accessing specialist mental health care involves passing through different levels and filters between the community and specialist care. This model highlights the importance of the primary care clinician in detecting disorder in presenting service users and acting as the gateway to the rest of the mental health care system through referral. The model has been adapted in recent years to accommodate a model of ‘stepped care’, through which the intensity and scale of intervention for the individual patient is linked to perceived need and severity of symptoms. It identifies the importance of self-management and lifestyle approaches for mild symptom presentations, and, because health systems have finite resources, restricts initial access to more specialised services. It also promotes the use of intervention by both the existing primary care team and other therapists accessed beyond secondary care.

The evidence base however for the effectiveness of particular models of primary mental health care is limited (NZGG 2008).


More recently the ‘stepped care’ and pathways approach is frequently linked to the placement of common mental disorders – particularly depression – within a chronic illness paradigm. Developing initiatives according to this model requires locating the disorder within a chronic illness conceptual framework, requiring the use of the chronic care management model (CCM) approach (Pincus et al 2001; Wagner 1998; Barr et al 2003; WHO 2002).

The CCM model highlights a systems approach, encompassing a number of key components, with a focus on the integration of care between health, social services and other agencies; integration between the different health sectors; and including case management for those in greatest need, utilisation of evidence, and appropriate information technology. It also reinforces the importance of self-management and appropriate monitoring (Myette 2008). Models of CCM that incorporate stepped care pathways have additional relevance for the PMHIs because they signal a potential change in the role of specialists and primary care nurses.

However, embracing the idea of primary mental health disorders as chronic and therefore requiring a CCM approach raises issues regarding the underlying nature and time course of common mental disorders (Gask 2005a; Pincus et al 2005; Klinkman et al 1997). The PMHIs also needed to provide care in which appropriate models could be matched to the clinical needs of service users within the available funding constraints.

A further important theme from the literature concerns the workforce implications for both existing and new staff in the primary care team. From a general practice perspective it is clear that there are a number of matters that constrain effective service utilisation, including (up until now) the lack of involvement of other primary health care professionals and, for GPs particularly, the lack of time to undertake effective mental health care work (Dew et al 2005).

The role of nurses and, in particular, that of the mental health co-ordinator/mental health nurse is of interest in the analysis of the PMHIs given the significant variation in how primary health care nurses (including practice nurses) have been involved in the delivery of mental health care (McKinlay 2002). Also, the increased use of psychological (talking) therapies and therapists represents a major potential change in the provision of primary mental health care in New Zealand. Until recently the evidence regarding the use of counsellors in mental health demonstrated only partial success in improving diagnosis, and even less in changes to management or outcome (Bashir et al 2000; Freidli et al 1997).

Despite this uncertain evidence base, workforce developments have continued in a number of countries with a view to increasing access to psychological therapies. The Australian experience is an important lesson given the relative similarities in the structure of general practice and payment systems. The Better Outcomes in Mental Health initiative was introduced there in 2001/02. Divisions of general practice in Australia were funded to establish programmes that allowed GPs, following training, to refer service users to psychological treatments. Service-user and provider satisfaction with these programmes was high, and clinical data showed an improvement in patient outcomes. Flexibility in the structure of the programmes and adaptation to local need were found to be important features (Winefield and Turnbull 2007; Vagholkar et al 2006; Thomas et al 2006).


1.5 The evaluation of the Primary Mental Health Initiatives A request for proposals for an evaluation of the initiatives was issued on 21 March 2005. The Ministry of Health defined the overall aims for the evaluation as being to:

• identify the extent to which the overall initiative had met the purposes of the funding and contributed to the Primary Health Care Strategy

• determine to what degree the initiatives had met the objectives of reducing health inequalities and improving health outcomes

• determine the effectiveness of the different types of service delivery for the client groups targeted

• specify the lessons or key elements that can be learned from the evaluation of the initiatives to inform the future development of primary mental health services.

The evaluation of the initiatives was intended to have a strong formative component, and to provide resources for the further development of primary mental health initiatives. It would also provide input into future planning and funding decisions. The Ministry of Health envisaged that the evaluation would cover:

• the ways the initiative developed

• a description of the target population

• the extent to which initiatives delivered on their aims and objectives, and on their contracts with their DHBs

• factors affecting the outcomes of the individual initiatives and the overall PMHIs

• analysis of the different types of service delivery and their effectiveness in meeting the mental health needs of the target group and identified sub-groups, including people with co-morbidities

• collection of a core set of quantifiable baseline data; for example, National Health Index, socio-demographic data, service utilisation, service delivery costs, referral pathways, mental health classification, co-morbidities and consumer co-payment data, from all of the initiatives; the emphasis was intended to be on using data sets already collected by PHOs and to reduce the burden for the PHOs in collecting and providing the same data

• case studies in a number of PHOs to gather more in-depth information on particular types of service delivery

• the lessons that can be learnt from the initiatives to inform the future development of primary mental health services, in particular the types of services that are effective for high-needs groups

• the ways that the initiatives improve integration between DHBs, mental health services, non-government organisations (NGOs) and PHOs

• service-user satisfaction with the initiatives

• service-user outcomes

• workforce issues.


An important guiding principle in the evaluation was to identify successful or challenging elements across initiatives rather than focus on individual PHOs. This meant that future development of the initiatives could be based on the experience of many different PHOs rather than just attempting to identify ‘flagship organisations’. An evaluation matrix was designed to assess key components of the initiatives. The main themes assessed were as follows:

• improving access for service users to: – the GP, practice nurse or other primary provider – other primary care services – secondary care

• liaison and integration of services: – within primary care – between primary and secondary care

• improving patient outcomes: – for those with problems of mild/moderate severity (the ‘target groups’, as outlined in the

original Ministry request for proposals to PHOs) – for those with enduring and severe conditions (the request for proposal response from

some PHOs identified particular groups of patients with severe and enduring conditions who they considered appropriate for the scope of these initiatives)

– increased detection of other physical problems

• education and training for: – providers – service users/patients

• decision support/IT packages: the provision and use of specific platforms or electronic tools for the initiatives

• service provision with a Māori/Pacific focus: Māori and Pacific groups were specifically targeted in the request for proposals, and the evaluation sought to assess the impact of kaupapa Māori and Pacific services, and mainstream providers with significant Māori and Pacific populations

• services for specific populations: although the great majority of initiatives focused on generic services for adults, some were specifically designed for particular populations, including: – age-specific (eg, services for youth) – alcohol/substance use.

This evaluation employed both quantitative and qualitative methodologies. Quantitative data enabled a descriptive study of uptake and utilisation of services. Qualitative methods were employed to provide individual and group insights from service users and providers into the structure, process and outcomes of the initiatives. Quantitative data helps to describe what is happening as a result of the initiatives, while the qualitative data helps understanding of why and how things happen.


This report contains both quantitative and qualitative data drawn from the period October 2005 to February 2008. It provides summaries of the activity in all of the 26 different programmes funded under the first phases of funding, as well as more detailed case studies on eight selected initiatives. The extended version of this report contains detailed information about all aspects of the evaluation process, including information about the specific areas evaluated as well as the full case studies of selected initiatives.


Chapter 2: Methods This chapter serves as a general overview of the methods used in this evaluation. For more details, please consult the extended report (Dowell et al 2008).

The evaluation used a mixed-methods approach, incorporating various types of quantitative and qualitative analysis. The aim in using this approach is ‘to obtain different but complementary data on the same topic’ (Morse 1997:122) to best understand the research problem. This hybrid approach, drawing on the strengths of different research approaches, is now commonly used in primary care and health services research, and in health service evaluations. Figure 2.1 at the end of this chapter illustrates how a mixed-methods approach is used to provide information aligned with both the evaluation aims and the initiative aims.

2.1 Qualitative methods A range of qualitative methods has been used, including individual, small group and focus group interviews. Individual interviews were generally undertaken when in-depth interviewing was required, including stakeholders, clinicians and service users.

Core research team members undertook all of the stakeholder interviews. Service-user interviews were undertaken by a consumer health services researcher, or a Māori health services researcher, or either of two members of the core research team. Purposeful sampling was undertaken to select the most appropriate participants depending on the research questions being answered (see in detail below).

Stakeholder individual, small group and focus group interviews A key informant sampling frame was used for selecting interviewees (Lincoln and Guba 1985). The main contact person for each initiative was the starting point for gaining access to key informants. This person was asked to nominate the most appropriate people to interview in order to capture information on:

• the overall strategic aims of the initiative – this group usually comprised any combination of the PHO manager, PHO mental health clinical champion and project manager

• how the initiative worked – this was usually the new or existing staff (mostly clinicians)4

employed to implement the project

• the perceptions and opinions of referrers – this group was a combination of GPs and practice nurses.

The semi-structured interview schedules were compiled on the basis of information gathered from service specifications and discussion on the nature of the initiative with the main contact person. Interview schedule compilation was an iterative process.

Several initiatives employed non-clinical staff to implement the project.


4

Case study methods Case study methodology is frequently chosen to report on an organisation, programme or process, or a combination of these (Gantley 1999). In the contracting phase of the evaluation the Ministry of Health requested that the evaluation incorporate eight case studies, which were undertaken to provide an in-depth view of particular types of initiatives. The case study sampling frame agreed between the Ministry and the evaluation team was as follows: • a Pacific initiative • an initiative adopting an integrated chronic care management (CCM) model • a small initiative lacking prior experience in primary mental health • a kaupapa Māori initiative • a medium-sized initiative • a large initiative in an advanced state of preparedness and with experience in primary mental

health • an initiative in a rural/semi-rural setting • a nursing-led service.

In addition to the routine qualitative data collected for all initiatives, additional data collection was undertaken for the case studies and included more frequent and in-depth face-to-face, email and phone contact. GPs, practice nurses and service users were also routinely interviewed for case studies.

Case study interviews, individual service-user data and document analysis were used to construct a narrative on each of the case studies, with specific reference to the following themes: • how particular types of service delivery worked in practice for different client groups and, in

particular, for high-need groups • the workforce and capacity issues that arose • service co-ordination and integration issues • the role of multidisciplinary teams and teamwork • service-user outcomes and satisfaction • the lessons that can be learned to inform future service development.

Service-user interviews A service-user perspective is a key component of evaluation research (Crabtree and Miller 1999). Service users were selected from each of the eight case study areas to allow for in-depth exploration of experiences.

A two-tiered recruitment process was instituted, whereby service users were invited to participate, via a letter sent out by local PMHI staff. Service users then self-nominated their interest to proceed by returning an ‘expression of interest’ form to the research team. When a service user posted back the expression of interest form, the interviewer made contact by phone to confirm their interest and arrange a time for an interview.

10 Evaluation of the Primary Mental Health Initiatives: Summary Report

One-off, in-depth individual interviews (generally face-to-face) were conducted with service users using a semi-structured interview schedule (see the extended report for details). A general inductive approach was taken to the analysis, with the aim of allowing meaning to emerge from the data (Kuper et al 2008). The overall aim was to allow the extensive and varied raw data from each initiative and between initiatives to be condensed and compared, so that commonalities and differences could emerge (Patton 1980, 2002; Pope et al 1999). A single member of the research team initially coded all the transcripts under common headings. The coded transcripts were then reviewed by a second member of the research team and refined as necessary (Liamputtong Rice and Ezzy 1999).

Document audit The original Request-for-proposal (RFP) documents were supplied by the Ministry of Health to the evaluation team, and each initiative was asked to supply other supporting data as the initiatives progressed. A variety of documents were gathered, including meeting minutes, staff manuals, client information and reports to the DHB.

Initiative documents were compared to the original RFP documents and in combination with the stakeholder interview data. The aim of the analysis was to look for coherence – or otherwise – with the original initiative aims, and for evolution of the initiatives as they progressed.

2.2 Quantitative methods

Individual service-user data As part of their contractual arrangements, the PHOs agreed to collect demographic data about individual service users. PHOs were also encouraged to support the collection of outcome measures from individual service users. To ascertain the variety of individual service-user data formats from the 26 initiatives, a stocktake of data was completed for early-start initiatives. Due to the varying IT capacity among PHOs, a decision was made to supply a standard data collection software package5 to those without existing collection templates, and to collect data in its original format from larger capacity PHOs who had already established their own data collection methods. The software was developed by a subcontractor, to a specification determined in conjunction with the evaluation team. Data sources were subsequently reduced to two types: standardised, which contains data in a form using the WSM tool, and non-standardised, which comes from all other initiatives not using the WSM tool.

The IT subcontractor mapped the data received from the initiatives back to the WSM data specification. The original data and the mapping transformations were maintained using SQL server software, and then exported to a Microsoft Access database format for analysis by the evaluation team. A central database was created, which combines the two different data types.

In some instances initiatives had not recorded, or did not supply, any information for a particular category (eg, no data may have been supplied on who referred the service user into the programme). Also, individual service users may have been missing data from one or more variables of interest (eg, a service user may not have any visit data recorded).

5 University of Otago, Wellington, data collection tool (WSM tool).


Analysis of most of the quantitative data was performed using Microsoft Access 2003 to select, group and count data across all service users. The analysis of screening tool scores was conducted using SAS 9.1 (SAS Institute, North Carolina, USA). These analyses used the SURVEY-based procedures (eg, PROC SURVEYMEANS and PROC SURVEYREG) to allow for the effects of clustering of scores due to differences between the initiatives.

Economic analysis A cost-effectiveness analysis explored the costs of service provision for a small number of initiatives against the outcomes yielded in terms of service-user numbers and mental health outcome scores (Kessler 10 and PHQ-9, see below).

A costing template was devised in conjunction with the Ministry of Health and a health economist from the University of Otago, Wellington. The template can be found in the extended report. A subgroup of four initiatives was asked to complete this costing template. Initiatives were chosen with the aim of capturing costs on a variety of different programmes, and also based on their capacity to provide this type of information.

Once the templates had been completed by the initiatives, the information provided was reviewed by an external health economist, and any inconsistencies, ambiguities or missing data were documented. The template was then sent back to the initiative for clarification. This process was repeated until the external health economist was satisfied with the quality of the economic data.

Information on the cost of the different types of service provision was taken from the costing template and compiled into a table. This was supplemented by contacting other initiatives and asking if any of their costs fell outside the existing cost range; if so, the range was extended to reflect these costs.

Outcome measures Clinical outcome scores were collected by the initiatives in several different ways.

The outcome tools used were the Kessler 10 (K10) (Kessler and Mroczek 1994), the General Health Questionnaire 12 (GHQ-12) and the Patient Health Questionnaire (PHQ-9) (Kroenke et al 2001). One initiative collected outcomes from service users using both the PHQ-9 and the Hospital Anxiety and Depression (HAD) scale (Zigmond and Snaith 1983) on each occasion. In order to standardise the analysis of these measures, only PHQ-9 scores for this initiative were included in the analysis.

To accommodate these three different scales, screening tool scores were converted from their native scales (K10, PHQ-9, GHQ-12) to a new ‘percentage point’ scale: that is, the actual score obtained was divided by the maximum score for that scale to give a percentage of the maximum possible score. For example, on the PHQ scale a score of 18 out of 27 translates to 66.67% of the maximum score on that scale. Since possible scores on the K10 scale fall between 10 and 50 (ie, offset by 10, with a range of 40), the calculation for K10 scores was: percentage point = (score – 10) /40.


There are no standardised methods for comparing scores across these different scoring tools when the data is collected from different individuals. One study has reported correlations in scores between the K10 and GHQ-12 scale scores completed by the same individual at the same time, reporting a rank-based correlation of 0.5 (Andrews and Slade 2001). The analysis of these outcome tool scores should thus be interpreted with some caution. The use of cluster-based analytic methods allows some control for the use of different scoring tools in the different initiatives, and provides more conservative estimates of variance (ie, wider confidence intervals) within the groupings used for the analysis of these outcomes (eg, the package of care model or diagnostic group).

To address the issue of whether improvements in score were maintained following the interventions, six-month follow-up scores were collected from a sample of service users in several initiatives. After data collection commenced, these initiatives were requested to supply the six-month data. This data was collected in a different database system to the main outcome data, so it was not possible to match these service users with their demographic data.

2.3 Talking therapy survey A survey of talking therapists and new primary mental health clinicians was undertaken in order to investigate aspects of the provision of counselling and psychotherapy as part of the initiatives.

The key contact person for each initiative was asked to supply a list of all talking therapists employed or contracted for the initiative. Mental health nurses, mental health co-ordinators or social workers who were involved in the initiatives were also added to this list because they were known to have an element of talking therapy in their interaction with service users. Surveys were posted directly to talking therapists, with a reply-paid envelope included. Respondents were asked to return the survey within two weeks. After this period had elapsed, respondents were phoned and prompted to complete the survey.

Survey data was manually entered from paper forms into an Access database. Analyses reflected the primary descriptive purpose of the study. Numerical data was summarised using measures of central tendency and variability, with appropriate aggregation up to meaningful groups. Free-form written accounts were printed out for visual inspection and coded into aggregate areas on the basis of empirical and theoretical interest. Broad thematic analysis of this material involved identifying the range of recurrent and divergent issues arising from the material.

2.4 Ethics approval The Multi-Region Ethics Committee approved the bulk of the study in February 2006, with subsequent approval to undertake service-user interviews obtained in May 2006, and approval to include a talking therapy survey and extend the evaluation to June 2008 obtained in April 2007 (ID number: MEC/05/12/177).


Figure 2.1: Alignment of aims and methods in a mixed methods evaluation of PMHIs

Evaluation aims

Identify the extent to which the overall Determine the Specify the lessons or key initiative has met the purposes of the effectiveness of elements that can be learned funding and contributed to reducing different types of from the evaluation to inform health inequalities and improving service delivery for the the future development of

health outcomes client groups targeted primary mental health

Improving access: • GP/PN • other primary • secondary

mental health

Liaison/integration: • within primary • between primary

and secondary

Improving patient

outcomes

Education and training of:

• providers • service

users

Decision support/IT packages

Māori/ Pacific service

provision

Services for specific populations:

• age-specific • alcohol/

substances

Evaluation methods

Quantitative data: service

users

Talking therapies

survey

Service-user interviews

Qualitative interviews on service delivery: • GP/PN • co-ordinators • talking therapists

Qualitative interviews of stakeholders:

• secondary mental health staff

• DHB staff • PHO staff

In-depth case

studies

Document review audit

Summary of interviews completed

Initiative aims

Table 2.1:

Initiative Interview/ meeting dates

PHO management

New primary mental health

clinicians

GPs/PNs Service users

Others

Northland December 2005 5 8 Advisory group (1) Cluster April 2007 3 6 Consumer rep. (1)

Project/clinical managers (2)

Combined Pacific

December 2006 May/August 2007

1 1

3 3 3

Secondary mental health nurse (1) Practice manager (1)

Procare Auckland

August 2006 February 2007

2 Project manager (1) Project manager (1)

Tamaki (Auckland)

January 2007 July 2007

Project manager (1)* Project manager (1)*

Tikapa Moana (Waiheke)

May 2007 1 1

Waitemata PMH

February 2007 Project manager (1)

Waitemata PLO

July 2007 Project managers (2)

East Health (Auckland)

April 2006 February 2007

1 1

1 1

People’s Healthcare Trust (Auckland)

February 2007 1 1 1



PHO management


clinicians


Others

Counties Manukau CCM Dep

April 2006

December 2006

February 2007 July/September 2007

3

1

8

2

1 4

7 DHB (3), project managers (1) Secondary mental health nurse (1) Project manager (1) DHB (3)*

Kawerau July 2007 December 2007 February 2008

1

1

1

2 1 1 Referrer to service (1) Referrer to service (1)

Pumau Ngāti Porou

March 2006 April 2007

1 1

1 2

1 focus group 1 focus group

6 Consumer rep (1) DHB (1), secondary MH (1)

Taupo May 2006 July 2007

1 1

2 4 DHB (1), secondary

MH (1)

Hawke’s Bay August 2006 1 1 Co-ordinator (1), project manager (1), talking therapists (2)

Taranaki August 2006 May/June 2007

3 2 11 8 Practice manager (1), DHB (2)

Whanganui March 2006 April 2007

2 1

5 6

2 focus groups 3

Secondary MH health (1)

Otaki February 2006 March 2007

1 1 1

Clinical leader (1)

Primary Solutions (Wellington)

February/July 2006 August 2007 February 2008

1 4

5

4

4

10 Clinical leader (1)

Talking therapists (2)

Tu te Wehi (Porirua)

August 2006 December 2007

1 2 2

SECPHO August 2006 September/ December 2007

1 2

1 1

Project manager (1)

Mid Valley (Lower Hutt)

January 2006 March 2007 1

2 1 1

Administrator (1) Administrator (1)

Piki Te Ora (Lower Hutt)

August 2006 December 2007

3 2 2

Wairarapa March 2006 February 2007

1 1

1 1

1 Clinical leader (1) Clinical leader (1), advisory group

West Coast May 2006 April 2007

2 2

2 focus groups 11

Rural Canterbury

May/November 2006 April 2007

1 2

2

Project manager (1)



PHO management


clinicians


Others

Mornington (Dunedin)

February 2006 February/April 2007

1 1 1

2 focus groups 5

7 Nurse manager (1) Nurse manager (1)*

* denotes phone interview Note: MH = mental health


Chapter 3: History Other than the provision of ‘usual’ GP-delivered primary mental health care, the majority of the PHOs involved in the initiatives did not have any formalised primary mental health programmes prior to the implementation of this new funding stream. Interviews with practice staff revealed a number of existing issues with addressing the mental health needs of their service users.

For a start, practice staff were often reluctant to raise the issue of mental health because they perceived there were very few satisfactory services or treatment options to offer the service user. Usual care options involved prescribing medication and/or counselling. Accessing counselling was seen as problematic in that all the options had drawbacks. Access via secondary mental health services involved meeting strict eligibility criteria and in some cases long waiting lists. A number of NGOs provide low- or no-cost counselling, but the quality of services was unknown and feedback was rarely received. Private counsellors are costly, and again quality was unknown and counsellor feedback was not often received, even if the referral came from a member of the practice team.

Practice staff, predominantly GPs, attempted to deal with mental health issues as best they could. Those who had experience in this area felt comfortable with treatment management, but those who were inexperienced or lacked confidence had no way to access good support or mentoring.

Overall, GPs and PNs lacked in-depth knowledge of how to access community and secondary mental health services appropriately and what services were available to access. Some had good back-up from secondary mental health services, but this seemed rare and generally communication/relationships with secondary mental health services were described as poor. This issue was also voiced by the secondary mental health services people interviewed.

There had been a number of previous mental health interventions in primary care, including pilot liaison schemes in Hawke’s Bay and Wellington (Rodenburg et al 2004), where liaison staff work for both primary care and secondary mental health services, and the longstanding use of liaison consultant psychiatrists in a low-cost primary care setting in Wellington. There was also some experience of the use of funded counsellors attached to general practices in Wanganui, and the option for GPs to use package-of-care funding for extended consultations and/or counselling in Hawke’s Bay. Procare in Auckland established a programme for high-needs service users (Māori, Pacific or low-income), where GPs could access funding for extended consultations and/or counselling. Some of the smaller PHOs had primary health-based problem gambling, sexual abuse counselling or social work intervention.


Chapter 4: Description of Initiatives: Models of Care This chapter is divided into two sections. The first part, ‘Introduction to the initiatives’, details the PHOs involved in the initiatives by DHB, whether or not PHOs combined, and if so how (see Table 4.1). It also covers what service types are provided in each initiative (see Table 4.2). The second part, ‘Models of care’, describes the different models of care that have been adopted by the initiatives and covers the advantages and challenges of the different models.

4.1 Introduction to the initiatives Table 4.1 summarises the initiatives, by DHB, from north to south, following the order used on the Ministry of Health website. Not all DHBs have a funded initiative in their region. The second column lists the name of the programme/initiative, the third column the PHOs involved, and the fourth column shows the name that will be commonly used throughout this report for each initiative.

Table 4.1: Primary Mental Health Initiatives, by DHB region

DHB Programme name PHOs Evaluation name

Northland Te Pou Ora o te Piringatahi: Northland Regional Primary Mental Health Initiatives

Hauora Hokianga Integrated Kaipara Care Incorporated Manaia Health Te Tai Tokerau Tihewa Mauriora Charitable Trust Whangaroa

Northland Cluster

Auckland Combined Pacific Initiative

Procare Auckland Initiative

Tamaki Healthcare PHO Primary Mental Health Initiative

AuckPAC Health Trust Board Langimalie Health Clinic Tongan Health Society Incorporated TaPasefika (TaPasefika Health Trust

Procare Network Auckland

Tamaki HealthCare Charitable Trust

Tikapa Moana (Waiheke Island)

Combined Pacific

Procare Auckland

Tamaki (Auckland)


Waitemata Primary Mental Health Options Waitemata

Primary Lifestyle Options (MH) Project

Coast to Coast (North Rodney) Procare Network North Ltd Te Puna Waiora Healthcare Trust

Harbour HealthWest

Waitemata PMHOP

Waitemata PLO


DHB Programme name PHOs Evaluation name

Counties Manukau

East Health Mental Health and Well Being Project

The Peoples Centre Trust Primary Mental Health Initiative

Counties Manukau Chronic Care Management Depression Programme

East Health Services

People’s Healthcare Trust

Procare Network Manukau TaPasefika (TaPasefika Health Trust) Te Kupenga o Hoturoa Charitable Trust




Bay of Plenty Youth Mental Health / Addictions Project

Kawerau Interim Kawerau

Tairawhiti Pumau ki te Oranga Ngati Porou Hauora Incorporated Pumau (Ngāti Porou)

Lakes Taupo Mental Health Initiative Lake Taupo Taupo

Hawke’s Bay Hawke’s Bay Initiative Hawke’s Bay Hawke’s Bay

Taranaki Taranaki Primary Connections Pinnacle Incorporated Taranaki Te Tihi Hauora o Taranaki

Taranaki

Whanganui Whanganui Early Intervention Mental Health Service

Whanganui Regional Whanganui

Mid Central Otaki Primary Mental Health Initiative

Otaki Otaki

Capital & Coast

Primary Solutions

Tu te Wehi Primary Mental Health Services

SECPHO Mental Health Initiative Project

Capital Kapiti Tumai mo te Iwi

Porirua Plus Ora Toa

South East and City Primary Health Organisation (SECPHO)



SECPHO

Hutt Valley Mid Valley Well Being Service Mid Valley Access

Piki te Ora ki Te Awakairangi

Mid Valley (Lower Hutt) Piki te Ora (Lower Hutt)

Wairarapa To Be Heard Wairarapa Community Wairarapa

West Coast West Coast Mental Health Programme

West Coast West Coast

Canterbury Mental Health in Primary Care Demonstration Model

Rural Canterbury Rural Canterbury

Otago Mornington Health Centre Mental Health Nurse

Mornington Mornington (Dunedin)

Table 4.2 shows the services that can be accessed through each initiative. For readability this table has been spread over two pages.


Table 4.2: Types of service available through each initiative

Initiative Talking therapist

contracted/ casual

Talking therapist

employed/ fixed

contract

Mental health

nurse/s

Social worker

Kaiawhina/ community

worker

Social support

Groups Extended GP /

nurse consults

Northland Cluster X X

Combined Pacific X

Procare Auckland X X

Tamaki (Auckland) X


X

Waitemata PMHOP X X

Waitemata PLO X X X X X X



X X


X X

Kawerau X X X

Pumau (Ngāti Porou) X X X X

Taupo X

Hawke’s Bay X X X

Taranaki X X

Whanganui X X

Otaki X X X


X X X

Tu te Wehi (Porirua) X X

SECPHO X X X


X X

Piki te Ora (Lower Hutt)

X X

Wairarapa X X X

West Coast X X X

Rural Canterbury X X


X X


Initiative Māori/ Pacific mental health worker

Mental health co-ordinator

(clinical)

Non-clinical co-ordinator

Psychiatrist Medication review

Alternative therapies

Youth able to access service

Northland Cluster X X

Combined Pacific X

Procare Auckland X

Tamaki (Auckland) X


X

Waitemata PMHOP X X

Waitemata PLO X (0.2 FTE) X (1 FTE)


X (GP/PN liaison role only

X



X X

Kawerau X X

Pumau (Ngāti Porou)

X

Taupo X X

Hawke’s Bay X X X

Taranaki X X

Whanganui X

Otaki X


X X X X


X

SECPHO X X


X X X

Piki te Ora (Lower Hutt)

Wairarapa X X

West Coast

Rural Canterbury



4.2 Models of care This section describes the different models of care that have been utilised for the initiatives. The 26 different programmes have been grouped into nine distinct model types. There are a number of ways the models could have been grouped, so a method has been chosen that aims to provide as much clarity for the reader as possible.

Each entry starts with a shortened description of the model type. This is followed by a list of the initiatives covered by this model type and a brief description of what is provided. Finally, the advantages and challenges unique to the model type are highlighted, based on the evaluation team’s knowledge of the initiatives and interpretation of the data. Common characteristics of models will not be highlighted (eg, the use of best available evidence).

For quick reference, the initiatives have been assigned to the following nine model types.

Model 1 initiatives: Northland Cluster, Taranaki, Primary Solutions (Wellington), Wairarapa, Procare Auckland

Model 2 initiatives: Waitemata PMHOP, Waitemata PLO, Hawke’s Bay

Model 3 initiatives: Tikapa Moana (Waiheke), People’s Healthcare Trust (Auckland), Pumau (Ngāti Porou), Otaki, Tu te Wehi (Porirua), SECPHO, Mid Valley (Lower Hutt), Piki te Ora (Lower Hutt), Mornington (Dunedin)

Model 4 initiatives: Tamaki (Auckland), West Coast, Rural Canterbury

Model 5 initiatives: Taupo, Whanganui

Model 6 initiative: Kawerau

Model 7 initiative: Counties Manukau CCM Dep

Model 8 initiative: Combined Pacific

Model 9 initiative: East Health (Auckland)


Model 1: Packages of care6 with contracted psychological therapies and clinical co-ordinators with or without social supports / extended GP consults

Initiatives using this model type Northland Cluster, Taranaki, Primary Solutions (Wellington), Wairarapa, Procare Auckland

Description Service users are assessed in person or by phone by a clinical co-ordinator, and, if appropriate, are offered a package of care involving one or more services.7 Funded services usually include a set or variable number of talking therapy sessions, and some initiatives also include alternative or traditional therapies such as massage. In some initiatives, packages of care were included for social supports, such as transport, child care or household maintenance, and/or extended consultations with the GP or practice nurse.

The co-ordinator’s role also involves mentoring practice staff, including arranging continuing professional development (CPD) sessions on mental health topics.

Advantages • Clinical co-ordinators have the time and skill to more fully assess service-user issues and then

select the most appropriate onward referral options. In some cases people do not receive a funded package of care because their needs are more appropriately met through community agencies or by referral to secondary mental health services.

• The clinical co-ordinator acquires knowledge of community services and supports.

• The clinical co-ordinator gains expertise in their role more quickly because the throughput is greater.

• Because the clinical co-ordinator is an experienced health professional, they can use their broad base of knowledge to undertake a holistic psychosocial assessment.

• There is the opportunity for practice staff to discuss options for referral with the co-ordinator.

• Co-ordinators familiarise themselves with talking therapy providers, and are therefore better able to match the service user to the talking therapist.

• A larger selection of talking therapists allows for some speciality in service provision (eg, grief, abuse issues, family therapy, relationships, kaupapa Māori).

• Practices are able to refer on quickly to the co-ordinator with a minimal increase in practice workload.

Disadvantages • It can create a big workload for co-ordinators.

6 ‘Packages of care’ is a term that should be interpreted with caution because it is used throughout the health sector in a variety of ways. A package of care in some situations may refer to just one intervention. ‘Packages of care’ in this case refers to a programme where there are usually a number of different services available.

7 With a registered health professional (eg, nurse, social worker).


• There is the potential for co-ordinators to be professionally isolated if they are not practice-based.

• Having appropriately trained clinical staff in the co-ordination role can be costly.

• Part of the co-ordination role involves administration. Without administrative back-up, highly skilled clinicians can spend significant amounts of time on administrative tasks.

• Having a large number of talking therapy providers can potentially result in quality monitoring issues.

• Co-ordinators may duplicate work already done in the general practice.

• Service users may not wish to tell their story again to the co-ordinator.

Model 2: Packages of care with contracted psychological therapy providers, non-clinical co-ordinators and extended GP consults

Initiatives using this model type Waitemata PMHOP, Waitemata PLO, Hawke’s Bay

Description Model 2 operates in a similar way to Model 1, but the GP assesses the service user and refers on as necessary for talking therapy provision and social supports.

The non-clinical co-ordinator focuses on administration and payment, and liaison between talking therapy providers, practice staff and service users. Some non-clinical co-ordinators were able to advise on community supports. They may also be involved in organising CPD sessions and developing resources for practice staff.

Advantages • Human resource costs are less for a non-clinical co-ordinator.

• The model encourages upskilling of existing general practice teams because they remain the primary care managers.

• Practice staff are required to interact more with talking therapy providers.

• The clinical responsibility for the service user clearly stays within general practice.

Disadvantages • Non-clinical co-ordinators may be forced to make judgements about clinical issues during

contact with service users.

• There is no in-depth, structured assessment by a skilled mental health clinical co-ordinator.

• There is the potential for variability in assessment and appropriateness of treatment options, depending on GP confidence and skill.

• It may be more expensive overall because there is no clinical co-ordinator to triage service users who did not need onward referral, or who could access alternatively funded services.


Model 3: A primary mental health clinician (PMHC)8 serving one or a small number of practices

Initiatives using this model type Tikapa Moana (Waiheke), People’s Healthcare Trust (Auckland), Pumau (Ngāti Porou), Otaki, Tu te Wehi (Porirua), SECPHO, Mid Valley (Lower Hutt), Piki te Ora (Lower Hutt), Mornington (Dunedin)

Description Because the PMHC is serving a small number of practice staff, more intensive work with service users is possible compared with some of the other models. In some initiatives, service-user contact is session limited (eg, up to six sessions), while in others there is no session limit.

GPs and practice nurses refer service users to the PMHC or discuss cases with him or her. Some service users will then be managed conjointly; some will stay with the GP or nurse, with the PMHC providing advice; and some will be solely managed by the PMHC.

Advantages • It facilitates a team-based approach within practices.

• It provides an identified person with whom practice staff can discuss individual service users.

• The PMHC does more intensive, one-to-one service-user work.

• A broad range of supports can be implemented in a more co-ordinated fashion.

• Service users can contact the key worker.

• The model allows for the tailoring and co-ordination of supports and services when the service user’s situation is complex.

• The co-location of the PMHC enhances referral as well as staff mentoring and in-service and peer review.

Disadvantages • This model relies on establishing excellent interpersonal relationships between clinician and

practice staff.

• It can only serve a limited number of service users.

• For most initiatives the PMHC is the sole provider. As a result, there is a risk of burnout because the PMHC has a potentially unlimited supply of referrals. Then, if the PMHC leaves, the initiative has to re-start relationships with a new PMHC.

• There is the potential for practice staff to refer all service users in the ‘too hard basket’ to the PMHC.

Nurse, social worker, counsellor.


8

• Practice staff have only one PMHC to refer service users to; if the relationship does not work, then the options are limited.

Model 4: Employed talking therapist(s) and GP liaison position (psychologist, psychiatrist or nurse)

Initiatives using this model type Tamaki (Auckland), West Coast, Rural Canterbury

Description A small number of talking therapists see service users for a limited number of sessions (usually six). The GP liaison provides mentoring or support to GPs and practice nurses, tailored to their prior knowledge and experience.

Advantages • There is more control over the quality of talking therapy interventions because small numbers

of therapists are employed.

• There are rigorous selection criteria for therapists, to ensure they work according to the specified psychological approach for the model.

• The professional development of talking therapists can be monitored.

• The referral pathway is simple and clear.

• There is more interaction between practice staff and talking therapy providers.

• GP liaison can mentor, peer-review, educate about other local mental health services, potentially broker access to secondary care, and build the capacity of local staff.

Disadvantages • Service users have limited options for talking therapy, potentially no specialty options, and

limited options if the first therapeutic relationship is unsatisfactory.

• The success of the model is dependent on the employed staff, with a need for considerable reorganisation if staff change.

• The model is personality driven.

Model 5: Multidisciplinary primary mental health team

Initiatives using this model type Taupo, Whanganui

Description Practice staff have access to a number of different disciplines for counselling and support (eg, social worker, kaupapa Māori counsellor, alcohol and drug, nurse). The multidisciplinary team


work closely together and in conjunction with practice staff; for example, all may attend practice multidisciplinary team meetings, or the GP, service user and primary mental health provider may have a case conference.

Advantages • There is a wide range of referral options for practice staff.

• There are inherent benefits in the multidisciplinary team approach (eg, shared problem-solving, multi-faceted interventions).

• It allows access to specialty options for talking therapy.

Disadvantages • It is more costly compared with other initiatives that use a less multidisciplinary approach.

• The referral pathway is more complex, and the wide range of choice for referrers may lead to uncertainty about optimal referral options.

Model 6: Youth mental health / addictions service

Initiatives using this model type Kawerau

Description Working within a non-governmental disability support service, this model involves a youth liaison role that engages with the wider community (schools, community groups, youth groups and community and specialist mental health teams) and uses a youth-friendly, whānau- and strengths-based model that accepts referral for assessment, one-to-one counselling and/or referral to an appropriate counselling/mental health specialist. The initiative has had limited interaction with general practice.

Advantages • It targets the needs of a very at-risk population.

• There is early intervention (seeing youth as young as 10 years old).

• Teamwork can be done with other key youth organisations, including education, police and social services.

• The community-based premises encourage youth access.

• The model is less clinically focused, and this is perceived as encouraging youth to access the service.

• Because it is culturally appropriate the service attracts high referral rates for Māori.


Disadvantages • It is not general practice-based and there is no specific GP input. As a result, it is possible

that wider health needs may not be identified.

• No health professional with a specific mental health services background is involved.

• No links have yet been established with primary care.

Model 7: Chronic care management model for depression

Initiatives using this model type Counties Manukau CCM Dep

Description Service users are able to access scheduled, extended consultations (up to 12) utilising the specific skills and activities available within the general practice, and have access to cognitive behavioural therapy. The service describes its entry criteria as service users with moderate to severe depression only (as measured by a PHQ-9 score of 15 and above – equivalent to a DSM IV major depressive episode). GPs and practice nurses have access to detailed practice manuals, workbooks and wellness plans, as well as decision support software that aids diagnosis and management.

Advantages • The model has a structured programme with very clear inclusion criteria, exit criteria and key

performance indicators.

• Data collection is integrated into the practice management systems.

• There is planned regular feedback on outcomes to the PHO and practice.

• The structured programme is easy for practice staff to follow.

• The programme is well supported by training modules.

• The programme is well supported by support staff.

Disadvantages • Inclusion criteria focus on depression and the entry PHQ-9 score, so service users cannot

enter the programme if they do not score sufficiently highly; or, even if they do reach the entry threshold, they may not score highly enough to be eligible for access to talking therapy.

• No social supports are accessible via the programme.

• There is limited targeting of high-needs service users (Māori, Pacific and low-income), although the programme is located in a geographical area of higher need.

• Although the programme was designed to encourage nurse involvement, most practices have only had nurses involved in a limited way.

• Rigid scheduling of appointments may not suit either the clinician or the service user.


Model 8: Use of psychiatrist / clinical nurse specialist and screening

Initiatives using this model type Combined Pacific

Description Service users are screened using a two-question screen during consultation, and then referred to either a clinical nurse specialist or a psychiatrist for ongoing care. GPs are marginally involved and practice nurses may do some screening. The psychiatrist builds capacity by mentoring practice staff and takes the responsibility for giving mental health care. This model operates slightly differently in the three different PHOs involved.

Advantages • The model is delivered in the Pacific-based practices, and so existing service users are more

likely to access it.

• Culturally appropriate service delivery helps to reduce stigma.

• The use of service users’ Pacific language assists with treatment, especially when talking about mental health concepts.

• There is the ability to link to family backgrounds.

• Active de-stigmatisation is occurring.

• Capacity building may be occurring.

• Different PHOs have added extra components, meaning the programme is sufficiently flexible to suit different PHO environments.

Disadvantages • It is difficult to determine what is attributable to the model type and what is attributable to the

PHO culture and who is delivering it.

• Moving between different Pacific cultures (eg, Tongan to Sāmoan) may be difficult for staff delivering the programme.

• It is difficult to extrapolate conclusions to other services: the model may only work within this environment.

Model 9: GP liaison role without a case load

Initiatives using this model type East Health (Auckland)

Description A nurse acts in a liaison role with general practice, providing support and advice to practice staff on management and referral options for mental health issues. The role also includes facilitating


better information flow between primary care and secondary mental health services, and mental health promotion in the community. Service users continue to be seen solely by GPs and practice nurses.

Advantages • This is a relatively inexpensive model to implement. • It encourages upskilling of practice staff. • Having no case load allows the co-ordinator to focus on mental health promotion. • There is referral to a wide variety of community-based services.

Disadvantages • No funded talking therapy or in-depth assessment is available for service users, although there

is referral to community-based talking therapy.

• Service users have access to usual care only – there are no add-ons.

4.3 Key points • Not all model types would work in all settings.

• Matching of the local model type to the local context appears to have been largely appropriate.

• Many models depend on having specific people in specific roles, which could lead to difficulties with sustainability.

• Models with well-developed IT support facilitate monitoring, reporting and feedback.

• Smaller initiatives and Māori and Pacific services have evolved a ‘specialist’ workforce that seems appropriate.

• New co-ordination roles (primary mental health clinician) have enabled increased access to community and social supports through increased sharing of knowledge.

• Models that give GPs a predominant role in assessment and resource management may not be optimal unless there is previous experience and expertise in primary mental health service development.

• Highly structured programmes may offer advantages in terms of clear managerial frameworks and tools, but their prescriptive nature may limit flexibility in terms of different client groups and problems.


Chapter 5: Case Study Synthesis This chapter is a summary of the case study synthesis that appears in the extended report. The aim of the synthesis is to compare and contrast the eight case studies and to draw together the main conclusions from these different initiatives.

5.1 The case studies The case studies were undertaken to provide an in-depth view of particular initiatives, with a specific focus on:

• how particular types of service delivery work in practice for different service-user groups and meet the needs of defined high-need groups

• workforce and capacity issues that arose

• service co-ordination and integration issues

• the role of multidisciplinary teams and teamwork in service delivery

• service-user outcomes and satisfaction

• the lessons that can be learnt to inform future service development.

The case studies agreed to by the Ministry of Health represented different types of service types, capacity and location. The evaluation team identified initiatives that broadly met the following criteria:

• a Pacific initiative: Combined Pacific

• an initiative adopting an integrated chronic care management model: Counties Manukau CCM Dep

• a small initiative lacking prior experience in primary mental health: Kawerau

• a kaupapa Māori initiative: Pumau Ngāti Porou

• a medium-sized initiative: Taranaki

• a large initiative in an advanced state of preparedness and with experience in primary mental health: Primary Solutions (Wellington)

• an initiative in a rural/semi-rural setting: West Coast

• a nursing-led service: Mornington (Dunedin).

Figure 5.1 places the case studies along a continuum, based on the level of input of existing primary care staff. At the far left of the continuum, existing primary care staff undertake a significant component of the clinical work and retain a case management position. At the far right of the continuum there is little or no input from primary health care services and the primary care team, and services are delivered by a community agency that does not have health care as its prime focus.


Figure 5.1: The eight case studies on a continuum, based on the type of input

GP/PNdelivered, with electronic decisionsupport +/cognitivebehaviouraltherapy

GP/PNscreen,and refer toco-ordinator for

counsellingandsupport

GP/PNscreen,and liaisonnurse mentors reappropriatecare

Mentalhealth specialistdelivered; mentoringGP/PN

Community agency based;no primary care input

Counties Manukau TaranakiPrimary West Coast MorningtonPumau Kawerau CCMDep Solutions (WG) (NgâtiPorou)

CombinedPacific

Although this chapter addresses themes derived from an assessment of the eight case studies, they reflect many issues common to all the initiatives. The following bullet points highlight many of the important factors that were found to optimise implementation of the initiatives.

5.2 Factors optimising implementation of the initiatives

Service delivery • Allow for adequate preparation time.

• Good contact with referring staff is essential.

• It takes time to build a service and learn who fits the service.

• Develop a definition of primary mental health conditions that includes life complexity and is not medicalised.

• Develop a language to talk with service users about mental health problems/issues without scaring them away.

• Both structured and unstructured services may work, depending on the context.

• Complicated package-of-care models may have too many choices to be easy to administer, or for GPs/PNs to feel knowledgeable about.

• High-needs populations were generally targeted. The ethnicity comparison with the PHO enrolled population showed a proportional rate in many – but not all – case studies. Although initiatives seemed willing to target high-needs service users, it appeared they did not always know how to do so.

• There was some reluctance by general practice teams to target Māori and Pacific service users ahead of other service users.

• Privacy is a significant concern for all involved, and may inhibit people from seeking help for mental health issues, particularly those living in rural settings or Pacific peoples.

• There may be a need for a completely different model of initiative if the population is high needs, hard to reach or has unique characteristics.


Workforce and capacity • The majority of practice staff preferred to have the specialist primary mental health clinician

available to do the more intensive time-consuming work, simply because they did not have the time, expertise or knowledge of community resources.

• Those practice staff who were doing the majority of the work found that although they were comfortable with this and enjoyed it, it did have an impact on their other work (eg, less time to see their other service users).

• Professional development needs for primary mental health clinicians were varied, although a common theme was training on brief, cognitive behavioural therapy-type interventions, and, for those from secondary care backgrounds, the nature of primary health care delivery.

• Practice staff in general valued learning about primary mental health management through a case-based learning approach; that is, the primary mental health clinician mentoring them through the management process.

Service co-ordination and integration • Co-ordinators who could deal with the complex social needs of service users were beneficial.

• Integration with secondary care is important. The key factors enabling this are communication and an understanding of the respective roles of the primary and secondary services.

• A model of integrated care that includes health, education, justice and corrections, and social services placed outside primary care may be appropriate in particular circumstances. An example would be youth services, where this holistic approach can be beneficial.

Teamwork • The model of care adopted influenced the complexity and effectiveness of teamwork. Some

initiatives only involved GPs and psychologists, while some involved GPs, PNs, mental health co-ordinator-type roles, counsellors and others. One initiative involved a variety of non-primary care sector professionals. There is a greater skill set available to the service user with larger teams.

• Initiatives that were co-located in general practices appeared to positively influence teamwork, both within practices and between general practice staff and initiative staff. Other models, whereby initiative staff visit general practices or provide regular and responsive support, also resulted in functional teamwork between those involved.


Chapter 6: Workforce This chapter outlines the workforce implications of the PMHIs. It contains a description of the workforce engaged on the initiatives, discussion of workforce capacity issues and consideration of the roles of both existing primary care team members and ‘new’ types of primary mental health clinicians.

6.1 Workforce description All initiatives have created new positions and roles, but the number of full-time equivalents (FTEs) established has varied greatly depending on the nature of the contract, the services being provided and the population served. Approximately 50 new FTE positions have been created across all of the initiatives, with 218 therapists retained on casual contracts to provide interventions.

The majority of initiatives have directed funding into new positions, often with only relatively small increases in time for existing general practice staff. Only three initiatives have not employed a primary mental health clinician, instead utilising existing practice staff to do the work.

Table 6.1 describes the most common positions that have been created and highlights some of the key components of these positions. Although the roles described have been given specific names, few were identical. For information on how initiatives used these different roles, the FTE allowance and a brief description of their role, see the extended document.

Table 6.1: Common positions created and their key components

Position (generic title) Generic description Qualification/training

Mental health nurse Mental health nurse (Māori /Pacific)

A combination of: • service development, including quality assurance

(QA) processes, policies and procedures, and project management

• assessment and referral on to appropriate services • an element of counselling • mentoring/upskilling existing primary care staff

Registered nurse, usually with experience and/or postgraduate training in secondary mental health services

Mental health co-ordinator (clinical)

A combination of: • service development, including QA processes,

policies and procedures, and project management • assessment and referral on to appropriate services • mentoring/upskilling existing primary care staff • an element of counselling

Registered health professional, usually nurse or social worker with experience and/or postgraduate training in mental health

Mental health co-ordinator (non-clinical)

This position involves: • service development • mentoring/upskilling existing primary care staff • oversight of QA.

Health sector management/ administration skills



Relapse prevention co-ordinator

Oversees the service user’s discharge from secondary mental health services back to primary care, and: • provides a link between secondary services, the

primary care team and the client/whānau • provides support to general practice teams • supports the patient/whānau with the transition back

to primary care • co-ordinates the relapse prevention plan.

Registered nurse with experience and/or postgraduate training in secondary mental health services

GP The GP: • refers to services. • screens for eligibility • has varied levels of input into care, from sole

clinician to a minor part in a multidisciplinary team • liaises with secondary mental health services • prescribes.

Medical degree and postgraduate fellowship of College of General Practitioners

Practice nurse The PN: • refers to services • screens for eligibility • has varied levels of input into care, from ongoing

input to a minor part in a multidisciplinary team • liaises with secondary mental health services.

Registered nurse

Pharmacist The pharmacist is available to do medication reviews for service users referred.

Registered pharmacist with special interest in psychiatry

Therapeutic recreation therapist

Uses recreation services to improve or maintain physical, mental, emotional or social functioning. Interventions include structured activity focused on symptom reduction; education; stress reduction and management; health maintenance; and community functioning and integration activities, including adventure/challenge activities and family interventions.

Bachelor of Health Science (Therapeutic Recreation)

Psychologist Brief intervention counselling utilising 4 to 6 sessions. Master in Clinical Psychology

Counsellor Counsellor (kaupapa Māori)

Provides counselling. Acts as referral agent to other services.

A variety of qualifications (see the extended report)

GP liaison/consultant Clinical psychologist

Provides mentoring/support for practice staff with regard to managing mental health issues with the service user.

Registered nurse with experience and/or postgraduate training in mental health Registered clinical psychologist

Clinical director/leader/ project co-ordinator

A person employed at the beginning of the project to set up services/systems etc and arrange contracts with providers.

Experience in project management and/or setting up previous services

Kaiāwhina/community support worker

This position: does group work organises social visits, practical needs, child care, respite, etc

Non-clinical role. The people in these positions have both been social work students.

Child and youth co-ordinator

A similar role to the mental health co-ordinator/mental health nurse, but with focus on youth only.

Previous experience in working with youth



Contracted providers Provides specialist service as and when required, sometimes chosen by client.

Psychologists, counsellors, massage therapists, kaupapa Māori counsellors

Psychiatrist The psychiatrist provides: • assessment and referral on to appropriate services • mentoring/upskilling existing primary care staff • an element of counselling.

Medical degree and postgraduate fellowship of College of Psychiatrists

Mental health co-ordinator / mental health nurse In many initiatives a significant new position and role has been that of a ‘primary mental health co-ordinator’ or ‘primary mental health nurse’. These positions have largely been occupied by nurses (in approximately 20 initiatives), but also by social workers (approximately five) and counsellors. Fifteen initiatives used this general role.

Although those people in this position have come from a variety of backgrounds, they have adopted a core range of skills and tasks in their role, as described below. Many of these primary mental health co-ordinators also carry a case load and use a variety of psychological and other therapeutic techniques.

The typical primary mental health co-ordinator/nurse A person taking this role typically had the following background. Coming from secondary community mental health nursing for the last 10–12 years, the co-ordinator is in mid-career and applied for the co-ordinator role because of a wish for new challenges and some frustration at current working conditions in secondary care. S/he has taken six months to develop links with a small network of local counsellors and therapists and has provided a protocol for referral from GPs and PNs.

S/he also handles a small case load of more complex service users and offers brief problem-solving therapy for some clients. As well as managing referral pathways, s/he is responsible for the administration of the initiative and is developing training sessions for the primary care teams. S/he is employed full time.

The nurse co-ordinator may be involved in: • needs assessment and service co-ordination for service users • mentoring practice staff and undertaking interdisciplinary reviews • building/strengthening networks between primary and secondary mental health services • advocacy for service users • providing advice to practice staff on referral options • case management • counselling • project management, involving service set-up (designing and implementing policies,

procedures, forms).


Non-clinical co-ordinator Three initiatives have put the majority of funding into increased time for existing staff (usually GPs and PNs), with varying degrees of FTE allowance for a co-ordinator (usually non-clinical), who functions more in a project management role without a case load. This non-clinical co-ordinator may be involved in other projects for the PHO, but in some PHOs works exclusively on the initiative.

The work of the non-clinical co-ordinator can involve any of the following: arranging and/or delivering mental health or systems training for practice staff, liaising with and managing providers of therapy, financial management, and reporting and liaising with practice staff on client progress.

Counsellors and psychologists Counsellors and psychologists have been employed in two ways: either directly by the PHO as an employee, or contracted on a fee-for-service basis. The fee-for-service model involves providers solely engaged in talking therapy intervention. Those employed by the PHO, as well as providing talking therapies, may also be involved in service development, mentoring practice staff, supervision of other staff, providing training, and acting as a referral conduit to other agencies.

In-depth assessment of the characteristics and work of the psychological therapists employed and contracted is provided in the talking therapies section.

6.2 Synthesis of commentary on the different roles An important feature of the development of the primary mental health workforce through these initiatives has been the variety of positions that have emerged. The new roles were based around the local needs of the PHO’s population, and on skilled staff being available. The final working arrangements of the new staff roles were dependent on the funding available. Interpretation of what constitutes the ideal workforce configuration will therefore differ depending on the funding still available and the needs of the PHO population. The initiatives have provided a number of different role configurations to inform future developments.

In areas where GPs and PNs are already working to capacity, and/or feel unsure of their ability to deliver these new models of primary mental health care, and/or want support from (or to share the load with) an experienced and skilled mental health clinician, a clinical co-ordinator role is preferable. In areas where there is sufficient time and infrastructure to assess, deliver and co-ordinate other input, a non-clinical co-ordinator role could be used. This is likely to be more efficient where there are larger volumes of service users.

Most initiatives found recruiting fairly straightforward and had a reasonable pool of appropriate applicants to choose from. Exceptions were more rural or isolated areas and when recruiting for practitioners identifying as Māori or Pacific. Potential workforce gaps in the existing models include community workers, administrators, access to funded counselling, and easily scheduled/ formalised access to specialist psychiatric advice. There is also a workforce gap in staff skilled in working with children and youth.


The potential for providing further on-the-job training and/or distance learning is significant and remains largely untapped. As the primary mental health workforce grows there will be a need to formalise training expectations and requirements. Some universities offer postgraduate distance papers in primary mental health, and other postgraduate programmes in mental health are available (see the extended report for further details). There is also an increasing range of published literature available about primary mental health issues.

A common theme in primary care is the value of a case-based learning model as opposed to the traditional continuing professional development (CPD) session, and this was commented on frequently by staff involved in the PMHIs. They have found it helpful to discuss particular case management options with new primary mental health specialists. If this model of training is promoted then it will be worth providing training to mental health specialists in the use of adult teaching principles. Teamwork has been achieved to varying degrees in the initiatives, but there is the potential to further enhance this through interdisciplinary training or education.

From a workforce perspective, overall the initiatives are running successfully in terms of service users seen and initial results achieved. There are, however, certain issues that will require further consideration. These include workforce needs in relation to the duration of intervention and the merits of specialist positions. There are implications for the existing primary care teams in terms of the level and type of involvement from practice nurses, having access to time with practice staff for mentorship, and balancing overall workload.

6.3 Implications for the development of a primary mental health care workforce

Appropriate match of skills to service model configuration The present configuration of the PMHIs envisages a variety of different types and models of initiative. Services have developed in many different contexts, and in particular there is a trade-off when it comes to the optimal size for an initiative’s organisation.

Evaluation of the PMHIs indicates that variations of stepped-care approaches are in use. Many less complex cases are managed by the existing primary care team, with more complex cases referred to a trained co-ordinator or a generic counsellor to undertake brief interventions. Further refinement of these models will facilitate more appropriate skill and resource allocation. When working with children and youth, the model needs to take account of working with staff from other sectors (eg, education), who may use other paradigms of thinking about primary mental health problems

Clinical skills development and maintenance The balance between the upskilling and development of existing primary care team staff and the employment of new mental health professionals is clearly still evolving. The initial format has in most cases been to initiate a service using new referral pathways, often using the new mental health professional roles. From the quantitative data it can be seen that the use of GP extended consultations also constitutes a very significant proportion of overall PMHI activity.


Policy needs to be further defined in terms of developing ‘new professional’ roles and identities as compared to upskilling the existing primary care team. This has considerable training implications, and optimal development would combine a balance of both approaches.

An additional point from the survey on talking therapies is the evidence that talking therapists were modifying existing therapies and tailoring them to meet the needs of PMHI populations. (See Chapter 7 on talking therapies for further information.)

People taking up new positions in primary mental health should have access to formal professional development opportunities as well as regular formalised supervision, given that they are working largely in isolation from other specialist mental health professionals. An adjunct to this should be the opportunity to meet regularly as a group for the purposes of problem solving and information sharing.

Training options Tertiary education providers should be asked to respond to workforce development needs and establish accredited short-term training courses and/or postgraduate qualifications within their established academic frameworks. This clearly has funding implications, since most PMHI/PHO budgets do not have funding for this sort of training. Training courses would ideally be interdisciplinary and, depending on content and/or complexity, cover subject matter such as primary mental health care, integrated care, domestic violence, abuse, alcohol and drugs, cognitive behavioural therapy (CBT) and brief intervention counselling.

Table 6.2: Training needs identified by new primary mental health clinicians

Profession Training needs

Co-ordinators with nursing backgrounds N = 7

Postgraduate Certificate in Health Science CBT Counselling modalities, benefits rights training, assessing people for best match of counsellors What is primary mental health? CBT, dialectical behaviour therapy Formal talking therapy training (eg, CBT) Formalised psychotherapy training or similar Refresher on counselling therapies

Counsellor (social work background) N = 1

Increased knowledge of medications; specific techniques/interventions for specific diagnoses based on best international evidence

Counsellor (occupational therapy background) N = 1

Further CBT-specific workshops

Psychologist N = 1

Upskilling/updating on specific practical management tools for a range of conditions/disorders

Social workers N = 2

CBT course Postgraduate Certificate in CBT

N = number of respondents in each category


Demand In many initiatives, workload and funding requirements escalated as demand increased. Under the present targeted approach, programmes need to have clear definitions for thresholds and screening in order to triage out those who do not meet at-risk or low-income group inclusion criteria. Consideration also needs to be given to keeping to a sustainable number of therapy sessions by using brief intervention models. When designing programmes according to best practice, however, there also needs to be flexibility to accommodate longer-duration interventions in some situations.

Given these challenges and wider societal barriers to seeking mental health care, as well as the known effectiveness of early intervention in primary mental health conditions, it is clear that a public debate is needed about the wider availability of primary mental health care.

Workforce development The present PMHIs were intended as ‘demonstration’ projects, but because of the demand and the challenges for service users and providers in accessing other mental health care, they have quickly become an established primary mental health service with its own infrastructure and service expectations. However, there is currently relatively little consistency or equity in service provision, and clear gaps in what is available. At present, for example, a service user in one PHO has the option under the initiative of being referred to a counsellor, an alcohol and drug counsellor, a kaupapa Māori counsellor or a social worker. In another setting, the option is a PN with experience in mental health. While both of these contexts might be appropriate, they are certainly not equivalent.

Also, existing services do not always meet the needs of many population groups. For example, children and young people were not offered services in approximately half of the programmes, and no initiatives included tailored/specific services for older adults. There is unlikely to be a single model type which will address the workforce context for most initiatives and areas, and there will need to be an increase in resourcing if increased consistency and equity are desired.

6.4 Key points • All of the PMHIs have created new roles and positions in primary care, the most common

being that of a primary mental health co-ordinator/nurse.

• This co-ordinator role involves a combination of service development, assessment and referral, elements of counselling, and mentoring of existing primary care staff. In some cases non-clinical co-ordinators have been employed to provide a project management-type role.

• Many initiatives have also utilised counsellors or psychologists to provide psychological interventions.

• Staff working in co-ordinator roles have valued the autonomy they have been granted to develop the service in the way they wish, coupled with the ability to shape the roles according to community need and individual experience.

• The co-ordinator role has evolved over time.


• As outlined in Chapter 5: Case Study Synthesis, despite being resource-intensive, models where the co-ordinator is able to visit regularly with each practice and take a more active mentoring role appear to be more effective in building staff capacity, efficacy and teamwork.

• Initial recruitment into the PMHI positions was generally straightforward in urban and semi-rural centres.

• Difficulties occurred when recruiting for primary mental health care clinicians and talking therapists identifying as Māori.

• Over time, with increasing workload and service development, potential workforce gaps in service delivery models have been identified. These gaps include community support workers, access to clinicians or other professionals with specialist children and youth skills, and specialist mental health advice.

• Support issues for staff employed in new positions include the need for clinical supervision and mentorship.

• Many people in new positions have come from a secondary mental health services background, and adapting their clinical skills to the primary health care environment has been highlighted as an important training need.

• Training in time-limited brief interventions and facilitating case-based learning models are also seen as important areas.

• Further work needs to target areas such as deciding on the appropriate use of skills, developing and maintaining skills, exploring appropriate training options, addressing high levels of demand, and future workforce additions or development.

• It is clearly more efficient that as primary mental health care develops there should be opportunities to share project management resources, and for existing staff to mentor those in new initiatives.


Chapter 7: Talking Therapies in Primary Mental Health Care in New Zealand This chapter describes a focused investigation of aspects of the provision of psychotherapies as part of the PMHIs and involved a postal survey to all psychotherapists involved in the PMHIs. The importance of this area of interest increased as it became clear that little was known about the detail of providing psychological therapy in this setting. In what follows, people who provide talking therapies are called ‘psychotherapists’. The term ‘psychotherapy’ is used to describe psychological work that includes counselling but excludes occupational therapy, recreational therapy, physical treatments such as acupuncture or massage, and pharmacological treatments.

A postal survey of all 293 psychotherapists known to be involved in the PMHIs was undertaken in 2007. The final response rate was 67.5%. Postal surveys of health professionals are known to commonly yield much lower response rates than this, which can invalidate the findings. In this study the combination of assertive follow-up of non-responders combined with a high degree of goodwill from therapists in relation to the initiatives yielded a high response rate. We can therefore regard these findings with some certainty. Only one similar survey of psychotherapists has been published (Tantam 2006). This UK postal survey was sent to all 4005 psychotherapists registered by the UK Council for Psychotherapy, but the response rate was only 34%, meaning the results were not generalisable.

7.1 Key results of the therapist survey A total of 198 surveys were returned: of these, 40 respondents had seen fewer than five clients and so were not included in the subsequent analysis, leaving a total pool of 158 respondents. The most common descriptor for the psychotherapist job title was counsellor (53.2%), followed by psychologist (26%), psychotherapist (21.5%), mental health co-ordinator (8.9%), and much smaller numbers of other classifications. Further details of the survey are found in the extended report.

What clinicians say they do and what they actually do when with service users may be different (Mathieson in press). We were not directly observing the therapists, so we asked about techniques as well as theoretical models to provide a richer picture of the kind of clinical work being done and the kinds of therapists doing the work.

A high proportion of therapists did not indicate a key model or theoretical base for their work (52.5%), although the majority of respondents (91.8%) endorsed one or more therapeutic models from a pre-populated list. Table 7.1 shows the overall frequency of endorsement of the therapeutic models. Note that respondents could select more than one model. The data clearly indicates that the majority of therapists use an eclectic approach, most commonly grounded in a general CBT model.


Table 7.1: Counts and percentages of respondents using specific therapeutic models*

Type of therapeutic model Count % (n = 145)

Behavioural/cognitive 118 81.4 Problem solving 86 59.3 Person centred 81 55.9 Psychoeducation 63 43.5 Transactional analysis 47 32.4 Family/couple therapy 45 31.0 Systemic 44 30.3 Interpersonal 42 29.0 Humanistic 39 26.9 Art therapy 36 24.8 Gestalt 34 23.5 Existential 33 22.8 Dialectical behaviour therapy 26 17.9 Cognitive analytic 23 15.9 Integrative 22 15.2 Object relations 22 15.2 Narrative therapy 20 13.8 Self-psychology 18 12.4 Neuro linguistic programming 17 11.7 Psychoanalysis (Freudian, Kleinian) 15 10.3 Transpersonal 14 9.7 Solution focused 12 8.3 Analytical psychology (Jungian) 10 6.9 Hypnotherapy 10 6.9 Play therapy 10 6.9 Personal construct theory 9 6.2 Strength-based 8 5.5 Psychosynthesis 7 4.8 Body therapists / body work 6 4.1 Hakomi 6 4.1 Acceptance and commitment therapy 5 3.5 Psychodrama 5 3.5 Bioenergetic 2 1.4 Group analytic 2 1.4 Music therapy 1 0.7 Other 34 23.5

* Respondents could tick more than one box.


7.2 Kaupapa Māori-informed models One of the main aims of the initiatives was to address the needs of Māori and Pacific peoples, so participants were asked to identify the main kaupapa Māori-based framework they used, if any. Table 7.2 shows the frequency with which each option was endorsed.

Table 7.2: Counts and percentages of respondents using kaupapa Māori models*

Kaupapa Māori model Count % (n = 158)

Te whare tapa whā 56 35.4 Dynamic of whanaungatanga 18 11.4 Tikanga 17 10.8 Te wheke 15 9.5 Tika, pono, aroha 15 9.5 Poutama pōwhiri 4 2.5 Kahui ao 2 1.3 Approaches for Pacific peoples** 2 1.3 Āwhiowhio 1 0.6 Basic respect for appropriate culture and family system 1 0.6 Ihi, wehi, wano 1 0.6 Maherehere 1 0.6 Pōwhiri process 1 0.6 Wairua 1 0.6

* Respondents could tick more than one box. ** The survey did not ask about this, but two respondents added this information.

For readers unfamiliar with these terms, an explanation is provided below.

• Te whare tapa whā: the four-sided model, which applies across all domains of health. The four domains are te taha hinengaro (psychological), te taha wairua (spiritual), te taha tinana (physical) and te taha whānau (family).

• Dynamic of whanaungatanga: support of whānau.

• Tikanga: the binding essence of the conduct of daily life; essential integrity.

• Te wheke: the octopus, incorporating waiora (total wellbeing for the individual and whānau), wairuatanga (spirituality), taha tinana, taha hinengaro, whanaungatanga, mana ake (uniqueness of each person and whānau), mauri (life force), ha a koro ma a kui ma (breath of life from forebears), and whatumanawa (open and healthy expression of emotion) (Pere 1984).

• Tika, pono, aroha: correctness, honesty and compassion, representing integrity with respect (Counties Manukau District Health Board 2006).

• Poutama pōwhiri: journey to reach higher potential; a system for growth, healing, learning and social development (Herd and Richards 2007).

• Kahui ao: healing through tikanga.


The majority of clinicians were non-Māori, but clearly a high proportion of them had at least some degree of familiarity with Māori models of health and mental health. Two participants mentioned models specifically for Pacific peoples: fale fono (‘blending with village identity and the spiritual dynamic of a personal persona’), lalaga and fonofale.

7.3 Other results of the therapist survey Other results are described in detail in the extended report. In summary, the most common therapeutic techniques used were cognitive, empathic attention, externalising the problem and future-orientation. Few therapists reported making major adaptations to their therapeutic techniques or adopting new models to work with this client group, but those that did adapt cited the required brevity of therapy as the major reason for doing so. The effect of this was compounded by the need to use therapy time to do further clinical assessment: a number of therapists considered that the referring GPs had not done sufficiently thorough psychosocial assessments.

7.4 Discussion A theme that arose here – and in other parts of the evaluation – is that the nature of primary mental health care is poorly understood beyond the knowledge that anxiety and depressive disorders are prevalent, and that there are complex barriers to problem disclosure (Dew et al 2005; Dew et al 2007) and access to treatment (MaGPIe Research Group 2005).

There is a lack of evidence about which specific components of psychotherapy make a difference to psychological outcomes in primary care. Considerations about theoretical integrity may (quite fairly) be a relatively low priority, as it might be argued that in the context of brief therapy, general therapist factors will be more relevant to effectiveness than complex techniques or theories. However, the wish to follow evidence was expressed by several respondents. The situation we observed was that therapists were doing their best and using their professional judgement, but that adherence to evidence was somewhat limited, partly because of the patchiness of the available evidence in this area.

This presents an issue for purchasers of treatment, as the current drivers favour a strongly evidence-based approach, and the evidence that does exist largely refers to models of therapy that are based on theory and technique, often derived from secondary mental health care perspectives.

Further challenges relate to workforce regulation and supply. Unlike other health professionals, until now not all psychotherapists have been required to be governed by the Health Practitioners Competency Assurance Act 2003. As a result there is variability in the mechanisms used by the various professional bodies which oversee their practice. This has been accompanied by a likely variability in quality. In addition, there are challenges in workforce supply and training opportunities for the range of psychotherapies in New Zealand.


In this evaluation a range of different psychotherapists and psychotherapies seemed to be associated with positive outcomes. Despite the need to take care not to fund ineffective or harmful therapies, as an interim measure there will have to be some diversity in what is considered acceptable to fund under any state-funded future schemes. In other words, some diversity will need to be accepted in the short to medium term while working towards some consistency of approach.

However, the principle of full choice that is articulated in the National Health Service (UK) – that the full range of therapies known to be effective should be offered to those in need (National Institute for Health and Clinical Excellence 2007) – may in the end be questionable in the New Zealand primary care setting, simply because of resource and workforce constraints.

7.5 Key points • The provision of opportunities to access brief psychotherapies from primary care is meeting a

large unmet need. The prevalence and extent of complex cases may have been underestimated at the outset of the initiatives.

• The approach taken to securing psychotherapy services varies according to the model of care used by the initiative and what psychotherapy resources are available locally.

• Psychotherapists are willing to engage with this kind of work, but would prefer stronger links with referrers in order to provide a high-quality service and maintain an appropriate sharing and management of clinical risk.

• The workforce engaged to do the work is highly varied in terms of qualifications, experience and type of treatment offered.

• For the majority of therapists the work associated with the mental health initiatives was a very small proportion of their working week. This raises the question of whether there should be a dedicated workforce with special expertise in working with the primary care client group. The high degree of overlap between clinical assessment and therapy would support such a development.

• There was some desire for additional training, especially in the types of therapy (particularly brief interventions) and core knowledge of mental disorders.


Chapter 8: Service-user Data Quantitative data has been obtained from all 25 initiatives required to supply service-user-level data, with some initiatives providing more data than others. The nature of the contracting process and how the initiatives were established meant that each initiative was free to choose its own data collection tool. The initiatives in turn were reliant on the primary care providers to supply some of the service-user data. Ensuring consistently accurate data collection across the hundreds of primary care providers in the initiatives was challenging. For the evaluation team, bringing together the various sources of data was a particular challenge.

8.1 Key points The full detail of which initiatives supplied data for each analysis can be found in the extended report. The variety of data collected, and the number of different formats in which data was supplied, has led to limitations in the evaluations team’s ability to analyse and interpret the data.

The final data set for the evaluation produced the following findings.

• A total of 16,007 services users were seen across the 25 initiatives that had started prior to the end of November 2007.

• More females than males were seen across all age categories, with the peak in this difference occurring in the 25–44 years age band.

• Of all service users, 69.3% were European, 17.5% were Māori and 4.3% were Pacific peoples. For Māori, this was a higher percentage than in the PHO-enrolled population. Closer analysis suggested that this pattern of higher access for Māori compared to the enrolled population was not consistent across the initiatives.

• The large majority of service users were referred into the initiatives through primary care services: either by a GP (86.9%) or a practice nurse (1.9%).

• Depression was the most common diagnosis: 69% of all service users were diagnosed with some form of depression.

• Co-morbidity data was not widely reported, but the majority of co-morbidities identified were related to mental health.

• A large number of different interventions were identified across the 53,001 visits made in total. General practice-based interventions – defined as extended GP consults – were the most common intervention received, with 49.7% of all service users making at least one visit for such an intervention.

• A wider group of talking therapies – defined by initiatives as brief intervention, counselling and CBT – made up the majority of interventions. The range for these therapies was 1 to 72 visits per service user. An upper quartile of six visits per service user for counselling services suggests that these types of visits consisted of multiple interventions.

• When the interventions received were investigated by service-user diagnosis, general practice-based interventions and counselling sessions were the most widely received interventions for each diagnostic group.


• Service users expressed satisfaction with the care given by the initiatives.

• Services for Māori included both kaupapa Māori and mainstream service options, and having both options available was perceived as optimal by Māori.

In summary, 16,007 service users were referred into the initiatives for mental health reasons. The majority of these received at least one intervention prior to the end of November 2007.

8.2 Demographics The demographic data was obtained from the service-user data supplied by the initiatives. Ministry of Health comparative figures were obtained from PHO enrolment figures as at January 2008.9 A total of 16,007 service users were seen in the 25 initiatives that supplied data for the evaluation. Age, gender and ethnicity data was available for the majority of these service users.

Figure 8.1 presents the age and gender data for the service users seen in the initiatives. For comparison, the overlaid symbols represent the gender and age distribution for the PHO enrolled population. The initiative count data are presented as solid bars and can be interpreted using the left-hand axis. The triangle and square symbols indicate percentages of females and males, respectively, in the PHO-enrolled population for each age grouping. The left- and right-hand axes are in proportion to each other, so that differences in relative distribution of age and gender between initiative service users and the general population can be observed. PHO enrolments in each age band over the age of 14 are higher for females than for males, although there are clearly proportionally more female service users in the initiatives.

Figure 8.1: Age and gender of service users compared with PHO enrolled population

Count of service users in initiative (columns) % of enrolled PHO population (markers) 6000

5000

4000

3000

2000

1000

0

Female Male Female (PHO enrolled) Male (PHO enrolled)

0–4 5–14 15–24 25–44 45–64 65+

Age range (years)

35

30

25

20

15

10

5

0

http://www.moh.govt.nz/moh.nsf/indexmh/phcs-pho-enrolment#demographics.


9

http://www.moh.govt.nz/moh.nsf/indexmh/phcs-pho-enrolment#demographics

Figure 8.2 compares the ethnicity of service users with the ethnicity of the New Zealand PHO enrolled populations, as recorded by the Ministry of Health at January 2008. It describes the identified ethnicity of service users for whom service-user demographic data on patient ethnicity and/or age was obtained. Classification of ethnicity data was performed at the initiative level (as detailed in Chapter 2: Methods).

Note that the category ‘Not stated’ includes both those service users who chose not to state an ethnicity (at the initiative level), and also those for whom ethnicity data was missing (when received by the evaluation team). The 16,007 service users represented in Figure 8.2 include all of those represented in Figure 8.1, as well as additional service users for whom age and/or gender data was not available.

Figure 8.2: Ethnicity of initiative service users compared with PHO enrolled population

Percentage of population 80%

70%

60%

50%

40%

30%

20%

10%

0%

2.0%

69.3%

17.5%

4.3% 3.4% 3.4%

7.5%

67.4%

13.4%

6.7%

2.5% 2.4%

Asian European Māori Pacific Not stated Other

Ethnicity group

Initiative population NZ enrolled population

8.3 Diagnoses

Major diagnostic groupings Service-user diagnoses are listed in Table 8.1 for the major groupings of diagnoses presenting in the initiatives. Information on diagnoses from 23 initiatives is included, representing a total of 14,384 service users. The counts presented in Table 8.1 are divided into major groupings (listed in bold) and sub-classifications. An individual service user can only count once towards each major grouping (eg, a total of 9926 service users presented with depression), but may have actually received more than one sub-classification within that major grouping (eg, a service user may be counted in both the depression sub-classification and the postnatal depression sub-classification if both of these diagnoses were recorded).


Table 8.1: Major diagnostic categories and sub-classifications

Diagnostic category Count % (n = 14,384)

Depression (any diagnosis) 9926 69.0 Affective psychoses 118 Depression 7713 Depression, mild episode 373 Depression, moderate episode 794 Depression, severe without psychosis 151 Depression / bipolar depression / unspecified depression 273 Postnatal depression 208 Suicide risk 171 Mixed anxiety and depressive disorder (also under anxiety) 377

Anxiety (any diagnosis) 3413 23.7 Anxiety state 3020 Mixed anxiety and depressive disorder (also under depression) 377 Post-traumatic stress disorder 18

Alcohol or substance abuse (any diagnosis) 744 5.2

Other major mental health disorders (any diagnosis) 1173 8.2 Ageing 7 Behavioural disorders 15 Developmental disorders 16 Eating disorders 57 Neuroses (non-anxiety / substance abuse) 705 Personality disorders 30 Psychosis, organic/non-organic 235 Schizotypal disorders 19 Sexual deviations or disorders 17 Somatisation disorder 80

Sub-threshold mental health or lifestyle diagnoses (any diagnosis) 3531 24.5 Adjustment reaction 1274 Anger management 16 Lifestyle concerns (weight issues, smoking, gambling) 36 Psychosocial 6 Relationship problems 1352 Sleep problems 81 Stress-related problems 1413 Other mental health (non-specific diagnosis) 185

Other diagnostic codes (eg, inappropriate/irrelevant codings) 1342 9.3 Assessment noted 336 Other 413 Other (no further definition) 601

The majority of service users (69%) presented with a diagnosis of depression or anxiety with depression. Table 8.1 presents the ‘any record of diagnosis’ count for each diagnostic category. Looking at particular combinations of diagnoses provided further information about the types of problems with which service users presented. The percentages reported below represent more complex combinations of diagnoses above the simple groupings shown in Table 8.1.


Altogether, service users with a depression, anxiety, or alcohol or substance abuse diagnosis (or any combination of these three) made up 78.8% of all individuals seen. The remaining 21.2% of service users did not receive one of these three major diagnoses. This meant that a sizeable minority presented with another major mental health diagnosis in the absence of any of the three most common diagnoses (6.3% of service users). Another reasonable-sized group presented without a diagnosis in either of the three major categories (depression, anxiety, or alcohol or substance abuse) or the other major mental health diagnosis group: thus, this group of 10.7% of service users only received diagnoses that were presented in the sub-threshold mental health diagnosis grouping.

Diagnosis by ethnicity Table 8.2 suggests that Māori service users were less likely to receive a ‘major mental health diagnosis’ than European service users. As a consequence of this, the likelihood of receiving a ‘sub-threshold mental health diagnosis’ was enhanced for Māori compared to European service users.

It is important to keep in mind that this evaluation should not be used as a measure of prevalence for mental health conditions. The population under consideration here is individuals meeting the criteria for treatment, and then receiving treatment, in the PMHIs. Individuals not presenting at primary health centres are therefore not sampled; likewise, individuals fitting a certain diagnosis (eg, depression) but not falling within the target groups for an initiative are not recorded in the data set, nor are individuals for whom services could be better provided (either in terms of quality of fit for client, or cost) through other services.

Table 8.2: Hierarchical diagnostic category, by ethnicity

Ethnicity N service

†

Exclusive hierarchical diagnostic group (raw %)

Major MH diagnosis

Sub-threshold MH diagnosis

usersMajor MH diagnosis

Other major MH

diagnosis

Sub-threshold MH

diagnosis

Other code*

Odds ratio**

95% CI Odds ratio**

95% CI

European Māori Pacific Asian Other Not stated

9764 2103 625 293 481 445

79.6% 73.2% 80.3% 87.0% 69.0% 77.8%

6.4% 5.0% 9.8% 3.8%

15.6% 0.7%

9.7% 17.5%

7.5% 6.8% 6.2%

18.4%

4.3% 4.3% 2.4% 2.4% 9.2% 3.2%

−0.68 1.04 1.65 0.57 0.89

Reference 0.61−0.76 0.85−1.28 1.17−2.34 0.47−0.7

0.71−1.12

−2.04 0.76 0.71 0.62 2.15

Reference 1.79−2.33 0.56−1.04 0.45−1.12 0.42−0.9 1.67−2.76

† Count of ethnicity group sizes for individuals with both ethnicity coding and one or more recorded diagnosis (total = 13,711).

* These were non-diagnostic codes recorded against individuals (eg, ‘Assessment noted’). They are displayed here so that each row will sum to 100%.

** An odds ratio less than 1 indicates that members of this ethnicity group were less likely to receive that diagnosis than the reference group (ie, European); similarly, an odds ratio greater than 1 indicates that members of the ethnicity group were more likely to receive that diagnosis than the reference group. If the confidence interval for a group excludes 1, then this difference was statistically significant compared to the reference group.


8.4 Interventions Table 8.3 lists all the interventions provided by the 23 initiatives for which this type of data was collected. Note that because service users could receive more than one intervention, the total number of service users listed exceeds the total number of service users providing data for interventions received.

This data represents interventions received by 15,571 service users (ie, those receiving at least one intervention). The list order in Table 8.3 represents the patient journey from primary care (the GP and practice nurse) to more specified interventions outside the primary care setting. The ‘Other (miscellaneous)’ grouping is broken down by sub-classification in the extended report; all of these individual ‘Other (miscellaneous)’ interventions were accessed by less than 1% of the total number of service users for which intervention data was available.

Table 8.3: Intervention visit types

Visit type No. of initiatives offering

visit type

Visits (n = 53,001)

Service users (n = 15,571)

Visits

Count % Count % Median Quartile range

Maximum visits

GP visits Extended GP consultation GP visit (not specific MH)*

12 11 1

15,440 13,670 1,770

29.1 25.8 3.3

7,744 7,189

555

49.8 46.2 3.6

1 1 2

1–2 1–2 1–4

77 77 20

Practice nurse follow-up 3 816 1.5 402 2.6 1 1–3 9

Initial assessment 11 2,999 5.7 2,821 18.1 1 1–1 16

Mental health nurse/ co-ordinator

8 3,099 5.8 1,204 7.7 2 1–3 25

Brief intervention 2 1,830 3.5 1,135 7.3 1 1–1 15

Counselling 15 16,354 30.9 4,069 26.1 3 1–6 72

Cognitive behavioural therapy 14 7,195 13.6 1,990 12.8 3 2–4 30

Other (miscellaneous) 13† 921 1.7 310 2.0 2 1–4 24

Other (no further detail) 13 4,232 8.0 684 4.4 4 1–7 142

* Data is from a single initiative: this is any GP consults by a service user seen through the mental health initiative during the period the initiative was running. Data could not be separated into mental health consultations and general consultations.

† The initiative offered at least one ‘other’ category of intervention.


8.5 Service-user perspective To gain an understanding of the impact of the interventions provided by the PMHIs it was decided to complete in-depth interviews with a number of service users. Service users were interviewed from the eight case study initiatives only. These case studies are: • Combined Pacific • Counties Manukau CCM Dep • Kawerau10

• Pumau (Ngāti Porou) • Taranaki • Primary Solutions (Wellington) • West Coast • Mornington (Dunedin).

This section is divided into two parts. Part One contains a summary of interviews with service users identifying as Māori. Part Two contains a summary of interviews with all other service users. Analysis of the service users who identified as Pacific is incorporated in Part Two.

Part One: Service users identifying as Māori The interviews analysed in this report from eight Māori service users provide a valuable insight into the experiences and needs of those using the PMHIs. The experiences described derive from a variety of living circumstances. Most Māori service users interviewed were women, and most were parents of at least four children. All presented with mild to moderate mental health issues, and all expressed high levels of stress. Suicidal thoughts or actions were common, and half of the Māori service users indicated a history of alcohol and/or drug use or abuse.

The high level of stress experienced by the Māori service users is consistent with research showing that the level of stress experienced by Māori as a population is disproportionately high. A number of social and economic factors contribute to this, including the effect of political reform (Brown 1999), institutional racism, colonisation and marginalisation. The competing roles and responsibilities that Māori women have in caring for others, and the pressures of living within severely limited incomes, have also been shown to influence this (Kiro 2000). Compelling international evidence on inequalities shows that people who live in disadvantaged circumstances have more illness, greater distress, more disability and shorter lives than those who are more affluent (Benzeval et al 1995). Stress is a major health problem, and, accordingly, a reduction of its incidence and impact is highlighted as an objective of the New Zealand Primary Health Care Strategy (Minister of Health 2001).

Following are the key points summarised from the Māori service users.

10 In Kawerau only one service user over the age of 16 years was able to be interviewed. The majority of service users were under 16 years and therefore required parental approval to be interviewed. This created difficultly for service users who did not want their parents to be aware they had sought treatment. An application was made to the Ethics Committee to interview those under 16 who did not wish their parents to be notified, but at the time the project ended, ethical approval had not been granted.


Most Māori service users were uncertain about how they initially made contact with the PMHI; for example, how they found out about it, who referred them to it, and what the main reasons were for this interaction. One of the reasons for this uncertainty seemed to be that Māori service users had often been involved with a number of organisations and services, and were simply not clear which one was the PMHI. This was more likely in the smaller provincial areas, where there was greater interaction and stronger relationships between services.

Most Māori service users reported being referred to their PMHI either directly or indirectly through a GP. A few had ongoing GP support, but most did not appear to have a strong relationship with their GP in this way. Three of the Māori service users were prescribed psychiatric medication by their GPs.

Precipitating factors varied and were specific to the individuals. However, at the time of their initial contact with their PMHI, all Māori service users were experiencing multiple stresses. Factors such as death in the whanau, sexual abuse and other illnesses led to extreme levels of stress, and these were found to be the main common factors.

In some cases these people required quite substantial support because of the multiple stresses they experienced. Holistic service provision was therefore required. This is an important finding for PMHIs. Despite being funded in a siloed way, mental health is perceived holistically by service users, and Māori mental health service providers typically deliver in this manner.

Because of the complexity of issues facing Māori service users there is a need for ongoing support and monitoring – whether directly for referral for more counselling, or in terms of linking back to the PMHIs for other services. Māori service users saw it as important for the PMHIs to ‘keep in touch’ with them after they finished accessing the services.

The significance of having Māori mental health service providers and a Māori understanding of health and wellbeing was mentioned. Two Māori service users identified the use of rongoā Māori, but did not indicate whether it was associated with their PMHI. Some of those interviewed indicated that PMHIs worked with their whānau indirectly. One noted that an important aspect of her PMHI’s service provision was its ability to provide a service that was tikanga-based. She suggested that this allowed her to connect and heal as a Māori. Another Māori service user was specifically referred on to a Māori counsellor and reported benefiting immensely from his cultural knowledge and expertise.

Most Māori service users were referred to counsellors by their PMHIs, and all but one of those referred received this service from a counsellor outside their PMHI. Others were assisted by their PMHIs with social support. Counselling sessions were commonly provided weekly and involved ‘homework’ for users to complete between sessions. The counselling was generally found to be very worthwhile. The majority described a good rapport with their counsellor, which allowed them to open up about issues that needed resolution.

A number of those interviewed referred to their first six counselling sessions being free, with funding needing to be found for subsequent sessions. This potential cost was a significant issue for many. All Māori service users indicated that unless the counselling was subsidised or free it would be inaccessible and a deterrent to their accessing support.


A number of lifestyle changes were identified as a result of accessing the PMHI. These included both physical and relationship changes, with greater communication and understanding of issues being a common outcome. There was no simultaneous use of secondary mental health specialists or secondary mental health services by any of the Māori service users at the time of their contact with PMHIs.

Distance and a lack of public transport were identified as barriers to access in rural communities. However, some felt that a lack of information about what primary mental health services were available rurally was a greater barrier than distance. A lack of recognition of the need for help was also identified. One Māori service user suggested there may be a greater need for services in rural communities due to the stress of living in an isolated environment.

The level of satisfaction with PMHIs was evident in the fact that almost all of those interviewed indicated they would not hesitate to seek assistance from a primary mental health service like the PMHIs in the future. They expressed relief and gratitude that they now ‘know where to go’. They recognised the importance of such support and had in place contingency plans should the need for further contact arise again. They were also very confident about referring whānau members to the PMHIs.

Suggestions for improvement were minimal because most Māori service users were very satisfied with the service they received from their PMHIs. One person did suggest that more advertising and information promoting PMHIs might be worthwhile, and another suggested more resources may be needed, particularly in terms of greater support for overworked staff.

Part Two: All other service users The interviews provided an important forum for hearing the service-user voice. Interview questions were designed to elicit a comprehensive understanding of service users’ needs and experiences and how well the differing PMHIs met those needs. The key points from these interviews can be summarised as follows.

Most participants had a complex range of issues motivating them to seek primary mental health care. These issues usually included a past history of psychological distress (with or without formal mental health care input), physical health problems, precipitating factors in their current life/family context, ongoing depression, anxiety, and unresolved post-traumatic stress disorders. People seemed to feel that pre-existing psychological issues had a greater overt impact than chronic or long-term physical conditions.

Most people were referred because they were experiencing the first presentation of what was formally described as ‘mild to moderate mental health illness’. A few were referred because they had enduring mental health issues but there was a long wait for secondary mental health services, and a small number were referred because of complex personal/family circumstances.

Successfully meeting service-user needs depended on adequate resourcing, both in terms of the number and skills of staff, and appropriate triage/assessment by either the primary mental health co-ordinator or the GP. The assessment needed to be comprehensive and holistic, and to include psychosocial issues. Thus, in some situations an effective team approach was required, one that addressed the service user’s practical and psychological issues.


The counsellor and the number of counselling sessions offered needed to be tailored to the individual participant. The provision of a free service was an important factor in service-user attendance. In some initiatives people felt they did not have sufficient information to make an informed choice of counsellor: sometimes the counsellor did not have the necessary skills and experience to meet their needs, or had not clearly identified their role, which led to unmet expectations. Unless the co-ordinator or GP maintained some supervision of the counselling process these difficulties were not resolved.

Greater service-user satisfaction was noted where ‘homework’ was given to promote reflection and/or to practise new life skills, and where the co-ordinator kept in touch by phone to ensure the person was making progress. Where there was a good match between service user and service provided, participants were effusive in their appreciation of what was being done with them.

Both appropriate cultural safety and role modelling when dealing with those of a different ethnic group are important facets of successful PMHIs. Such approaches may take a different form from the more traditional ‘Western’ approach to psychological health and wellbeing. An integral part of such an approach would be the focus on the use of non-Western ways of conceptualising and articulating mental health and illness and the importance of family models of care.

It is possible that additional health promotion needs to be undertaken before New Zealand men feel enabled to seek help for mental health issues.

For most service users, the PMHIs provided a timely, effective and positive solution to mild to moderate psychological problems.

8.6 Utilisation of data from HealthStat (authored by CBG Research)

As part of a separate piece of work the Ministry of Health contracted with CBG research to provide further information about the Primary Mental Health Initiatives. A summary of this work is included below.

PMHI HealthStat primary care data analysis To obtain further information about the health care experiences of clients of the PMHIs, data from the HealthStat national random sample of general practices was utilised. HealthStat collects a comprehensive anonymous data set from a random sample of over 10% of practices in New Zealand each week. The HealthStat panel is widely used in evaluation and monitoring work in the public sector.

In October 2006 40 HealthStat practices were participating in a PMHI and 63 were not. Practices that had been participating in a PMHI were then asked if they would be willing to supply the National Health Index numbers of 10 patients that had received a PMHI intervention in the period immediately after 1 October 2006. The primary mental health care experience of these patients was compared with a group of matched controls, taken from 24 practices that did not participate in the PMHI at all up to October 2007. Analyses of data were divided into periods of one year before and one year after, based on the actual date of the first intervention in the case of PMHI patients and on the date 1 October 2006 for the control group.


Data was collected for 310 PMHI patients. Patients in a PMHI were predominantly female, middle-aged and non-Māori non-Pacific. PMHI patients were generally well engaged with general practice, utilising services at least four times in the previous year in 70% of cases. A third came from the most deprived socioeconomic quintile, and 76% attended large practices (with over 5000 registered patients). Interestingly, 67% of patients who received a PMHI intervention had no previous recorded diagnosis of anxiety or depression.

The control group had more consultations than the PMHI group per annum in both the before and after periods. The number of consultations per annum after a PMHI intervention decreased in the PMHI group and increased in the control group after the set date. This difference was statistically significant. The fees paid by both groups did not differ, but dropped slightly in both groups over the observation period.

There were significant differences in prescribing patterns in the two groups. PMHI patients had fewer prescriptions written overall (70% of the control group numbers of prescriptions), and dramatically fewer prescriptions for antidepressants, with less than 10%, prescribing in the control group: there were 94 prescriptions in the control group and only nine in the PMHI group. The number of antidepressants prescribed dropped in the ‘after’ period in both groups, dropping from seven to two in the PMHI group.

Hospital admissions data were analysed, but the numbers were very small overall, with only three mental health admissions in total for both groups.

The data suggests that the patients in PMHIs were already receiving fewer prescriptions, and antidepressants in particular, than a matched control group. This may reflect targeting of PMHI interventions to patients not receiving or requiring antidepressants.

A one-page fax-back survey was conducted to see if practices that did not provide PMHI interventions differed from those that did, with data received from 87 of 97 practices. Practices did not differ by size or staffing (doctor or nurse FTEs). In general PMHI practices had made slightly more referrals to counselling services, and those services were cheaper, including more free services. Most of the high users of counselling services were in the PMHI group. However, many non-PMHI practices had access to a range of counselling services and utilised them at similar rates.

Analyses of diagnosis rates and prescribing across the entire HealthStat panel showed a peak in new diagnoses of depression around the time the initial National Depression Initiative advertising was screened on TV in October 2006. Rates fell after this but have climbed back to this level over the next two years. The prescribing of antidepressants has levelled off, indicating less prescribing of antidepressants to people with a recorded diagnosis of depression in the last year.

This concludes the summary of HealthStat information provided by CBG research.


Chapter 9: Outcome Measures In order to assess service effectiveness, each initiative was required to record a pre-intervention and post-intervention score on a mental health status assessment tool for every service user. When funding for the initiatives was first rolled out, the Kessler 10 (K10) was the recommended measure. However, in some cases other measures were used (eg, where a mental health programme was already in place).

The three common measures used were the Kessler 10 (K10), used by 17 initiatives (Kessler and Mroczek 1994); the Patient Health Questionnaire (PHQ-9), used by two initiatives (Kroenke et al 2001); and the General Health Questionnaire (GHQ-12), used by two initiatives. Other measures used include the Hospital Anxiety and Depression Scale (HAD), and the Case Finding and Help Assessment Tool ( CHAT). Some initiatives used more than one measure and four initiatives did not produce any outcome data. Further details on the scope and validity of these questionnaires are available in the extended report.

It should be noted here that all of these outcome measures were originally intended for use as screening tools rather than for measuring improvement or change over time. In addition, the analysis of pre-intervention scores and score improvement following intervention was complicated by several factors, including:

• the use of three different outcome measures across initiatives

• incomplete coverage of recorded scores – response rates varied considerably across initiatives, introducing the possibility of biases in the recording of improvement scores

• some initiatives had minimum and/or maximum scores for entry into the programme, which can influence average measures of scores and improvements.

9.1 Methods and analysis Because responses were collected from three different measures this data needed to be transformed prior to analysis. There is no formal or informal method for comparing scores on these questionnaires, so each score was converted to a percentage of the maximum score range on the scale used. The value of this transformed variable is referred to in the text as ‘percentage points’. To give an example, the K10 measure covers a 40-point range between 10 and 50 points. An individual who scores 30 on the K10 would be recoded as having a score of 50 percentage points (ie, a score of 30 falls at the 50% point between the minimum and maximum on this scale). Further details on the rationale for using this methodology are included in the extended report.

The analysis presented looks at pre-intervention scores as a measure of initial symptom severity, as well as improvement in score following intervention. The sub-analyses include looking at these measures by diagnosis and ethnicity.

The analysis of pre-intervention scores reported in the text and in Figure 9.3 is based on all service users with these scores, giving a total sample of 11,595 service users in 18 initiatives. The remainder of the figures in this section present data from individuals with both pre- and post-intervention scores recorded. This gave a sample size of 4263 service users over a total of 17 initiatives. The analysis of improvement scores in the text is based on this sample.


The statistical analysis controls for possible clustering of outcome scores by initiative (the tendency for individuals in an initiative to have similar scores), as well as accounting for variation in the average scores obtained in the different initiatives. This was achieved by including the initiative in which a service user was treated as a cluster variable in the analysis. Unless otherwise noted, data analysis was carried out using PROC SURVEYREG in SAS 9.1 (SAS Institute Inc., North Carolina, USA).

The average improvement in score over all the service users with paired pre- and post-intervention data was estimated using meta-analysis methods that are commonly used to combine statistical results across multiple independently conducted studies. This test is based on effect sizes calculated for each initiative. Full details of this methodology are included in the extended report. Note that an effect size of zero would indicate that, on average, scores did not change after the intervention. Cohen (1988) has recommended the guidelines of 0.8 as indicating a large magnitude effect, 0.5 a moderate effect and 0.2 a small effect. This meta-analysis was carried out with a random-effects meta-analysis method, using the Metan (Harris et al 2007) package running in the STATA software environment.

9.2 Overall scores Higher scores on all three scales used by the initiatives were consistent with greater levels of psychological distress. The average pre-intervention score for all 11,595 service users with this score recorded was 54.3 percentage points (95% CI: 46.8 to 61.9). This average score would correspond to 31.7 points on the K10 scale (95% CI: 28.7 to 34.8). Figure 9.1 presents the mean pre-intervention and post-intervention scores, in percentage points, for the 4263 service users with both scores recorded.


Figure 9.1: Mean outcome scores (as percentage points) for service users at pre- and post-intervention

Outcome score (percentage points)

0

10

20

30

40

50

60

70

80 Pre-intervention Post-intervention

Time of measurement

Note: Vertical lines represent the 95% confidence interval of the mean.

The average improvement in scores following intervention can be seen in the difference between the two bars in Figure 9.1, and in the non-overlapping confidence intervals.

A less biased estimate of the average improvement after intervention was calculated based on random-effects meta-analysis methodology. Figure 9.2 shows the data from this meta-analysis, with scores for each initiative recalculated as effect sizes. As noted in the methods, an effect size of zero would indicate that the mean improvement following intervention was zero. The mean improvement in each initiative is represented by the dot markers, with the horizontal lines representing the 95% confidence interval of the improvement estimate. At the bottom of the figure is a diamond, which represents the estimate of the average effect size over all initiatives supplying data for at least 10 service users with matched pre-intervention and post-intervention scores (a total of 14 initiatives, with a minimum of 40 service users): the centre of the diamond represents the mean estimate, and the horizontal width of the diamond covers the 95% confidence interval of this estimate.


Figure 9.2: Random effects meta-analysis of improvement across all initiatives (presented in random order)

1.42 (1.07, 1.77)

2.69 (2.23, 3.15)

1.65 (1.47, 1.83)

1.18 (0.89, 1.47)

1.24 (0.76, 1.71)

1.78 (1.41, 2.16)

0.14 (0.03, 0.25)

1.68 (1.19, 2.17)

1.46 (1.30, 1.63)

0.55 (0.42, 0.67)

2.33 (1.73, 2.94)

1.32 (1.20, 1.45)

1.27 (1.13, 1.42)

1.17 (0.89, 1.46)

1.72 (1.62, 1.82)

Effect size (95% CI)

Effect size 0 1 2 3-1-2

Notes: Black dots indicate initiative-level effect size (the grey horizontal bar is the 95% confidence interval). The diamond indicates the overall random-effects estimate of effect size (the width of the diamond indicates the 95% confidence interval).

The actual effect size calculated over the initiatives was 1.42 (95% CI: 1.07 to 1.77), which is large by Cohen’s standards. The results of this analysis can then be converted back to different metrics; for example, giving a mean improvement of 29.4 on the percentage point scale (95% CI: 22.1 to 36.6). In terms of the K10 scale, this mean improvement was 11.7 points (95% CI: 8.9 to 14.7). In all cases, the mean improvement in score was significantly greater than zero.

9.3 Individual variation The above summary, and the more detailed analysis in subsequent sections, deals with differences in mean pre-intervention scores or improvements following intervention. An equally important source of information lies in the distribution of the individuals’ scores: knowing that the mean improvement was greater than zero does not tell us what proportion of individual service users improved their score.

The vertical grey bars in Figure 9.3 show the distribution of all available pre-intervention scores from service users (note that no correction for clustering by initiative has been performed for this figure). The vertical black line shows the mean pre-intervention score (as reported above), with the black curve showing the approximate normal distribution for this data (based on the mean and standard deviation in this group). The lower and upper quartile scores in this cohort were 40 and 70 percentage points, respectively. In terms of the K10 measure, this means that half of all service users had pre-intervention scores in the range of 26 to 38 points.


Figure 9.3: Distribution of pre-intervention scores (as percentage points) across all service users

Mean

1400

1200

1000

800

600

400

200

0

Pre-intervention score (percentage points) Note: The vertical black line gives the position of the mean pre-intervention score in these service users, and the black curved line represents the approximate normal distribution for this data.

The distribution of improvements in scores is shown in Figure 9.4 (on the percentage point scale), across all service users with paired data. The vertical black line indicates the mean improvement for this sample (based on the cluster-corrected average rather than the meta-analysis estimate), and the black curve represents the approximate normal distribution for this data. The dotted vertical line indicates the ‘zero change in score’ point: individuals above this point had some improvement in their score following intervention, while those individuals falling below this point had a higher severity score following intervention.

Ser

vice

-use

r cou

nt

0 10 20 30 40 50 60 70 80 90 100


Figure 9.4: Distribution of improvement in scores (as percentage points) across all service users with paired pre- and post-intervention scores

Zero change Mean

Ser

vice

-use

r cou

nt

400

300

200

100

0 -80 -70 -60 -50 -40 -30 -20 -10 0 10 20 30 40 50 60 70 80 90 100 110

Improvement in score (percentage points) Note: The vertical black line gives the position of the mean improvement in score in these service users, and the black curved line represents the approximate normal distribution for this data.

It is possible to calculate the proportion of all service users who showed an improvement of a certain magnitude. These are shown in Table 9.1, where it can be seen that a sizeable majority of all service users (58.1%) showed an improvement of at least 20 percentage points following the intervention (equivalent to a change of 8 points on the K10 scale).

Table 9.1: Proportions of service users showing different magnitudes of improvement

Magnitude of improvement (percentage points)

Percentage of individuals showing improvement

Greater than 0 10 percentage points or more 20 percentage points or more 30 percentage points or more 50 percentage points or more

80.9 72.4 58.1 42.3 18.0


9.4 Outcome scores by diagnostic group Both pre-intervention scores and improvements following intervention were analysed using a diagnostic group as a comparison group. Because individual service users could receive multiple diagnoses in most initiatives, a hierarchical classification system was used. This hierarchy is summarised in Table 9.2, with the number of service users classified in each category noted. A service user was classified in the highest category for which he or she received a diagnosis.

Table 9.2: Number of episodes with pre-intervention scores, by main diagnostic group

Position in hierarchy

Main diagnostic group n with diagnosis and pre-intervention score*

n with diagnosis and pre- and post-

intervention scores*

1 2 3 4 5 6

Depression and anxiety Depression Anxiety Alcohol or substance abuse Other major mental health diagnosis Sub-threshold mental health diagnosis

1653 5865 747 153 723 845

655 2632 176 39

256 250

* Service users can receive more than one distinct episode of care, so this count refers to the number of episodes with a diagnosis and pre-intervention score rather than the number of service users.

Figure 9.5 shows the average pre-intervention and post-intervention scores for these six diagnostic categories, for those service users who had diagnostic information as well as pre-intervention and post-intervention scores.

Figure 9.5: Average outcome scores (percentage points) at pre-intervention and post-intervention, by exclusive diagnostic category

Outcome score (percentage points)

0

10

20

30

40

50

60

70

80 Pre-intervention Post-intervention

Depression Anxiety and Anxiety Alcohol or substance Other major mental Sub-threshold depression abuse health diagnosis mental health

diagnosis

Diagnostic group

Note: Vertical lines represent the 95% confidence interval of the estimate.


Pre-intervention scores were significantly different between the different diagnostic groups (statistical test result from regression model: F [5,17] = 17.4, p value for test of null hypothesis < .001). It is important to note that the analysis is based on all available pre-intervention scores, whereas Figure 9.5 presents pre- and post- scores only for those individuals who had both measures. The following is a summary of the significant differences.

Pre-intervention scores were significantly higher in the depression diagnostic group than in the anxiety and alcohol or substance abuse diagnostic groups (p values all < .021). Service users with a diagnosis of anxiety with depression scored higher than individuals with anxiety (no depression) or alcohol or substance abuse (p values all < .007). Pre-intervention scores in the depression and the anxiety-with-depression groups were not significantly different (p = .55); likewise, pre-intervention scores in the anxiety (no depression) and alcohol or substance abuse groups were not significantly different (p = .347).

Scores were higher for individuals with one of the four major diagnoses (depression, anxiety, alcohol or substance abuse, considered as a combined group) than for service users with a sub-threshold mental health diagnosis (p = .008). The difference between the main mental health diagnosis group (the four categories given above) and the other major mental health diagnosis group was only marginally significant (p = .085). The other major mental health diagnosis and sub-threshold mental health diagnosis group pre-intervention scores were not significantly different (p = .25).

The average improvement in all diagnostic groups was significantly greater than zero, as can be seen in Figure 9.5, where pre-intervention and post-intervention scores in each group are clearly significantly different. The mean improvement (change between pre- and post-intervention scores) was not significantly different between the six diagnostic categories: F (5,15) = 0.96, p = .47.

9.5 Ethnicity Pre-intervention scores and improvements in scores were also compared between the different ethnic groups. Figure 9.6 presents the pre-intervention and post-intervention scores by ethnicity, for all service users with complete scores at both time points. Analysis of pre-intervention scores was conducted on data from all service users with these scores. Comparison of confidence intervals showed that there were no significant differences in pre-intervention score between the six ethnic groups, as can be seen in Figure 9.6. The analysis of improvement scores also showed no significant difference in the magnitude of improvements between the ethnic groups: F (5,16) = 0.94, p = 0.48.


Figure 9.6: Pre-intervention and post-intervention scores (as a percentage of the scale maximum), by ethnicity

Outcome score (percentage points) 90

80

70

60

50

40

30

20

10

0

Pre-intervention Post-intervention

European Māori Pacific Asian Other Not stated

Ethnicity

Note: Vertical lines represent the 95% confidence intervals.

9.6 Six-month follow-up To address the issue of whether improvements in score were maintained following the interventions, six-month follow-up scores were collected from a sample of service users in several initiatives. These initiatives were requested to supply the six-month data. The small size of this six-month cohort is partly due to the fact that only a few initiatives had been running long enough to generate longitudinal data extending six months after treatment completion. This provided a sample of 110 individuals from four initiatives who had completed pre-intervention, post-intervention (ie, at the end of the treatment period) and six-month follow-up outcome measures. Figure 9.7 shows the average scores (as percentage points), with 95% confidence intervals, at pre-intervention, post-intervention and the six-month follow-up in this sub-sample.


Figure 9.7: Mean outcome measure scores (as percentage points) at pre-intervention, post-intervention, and six months after cessation of treatment

Outcome score (percentage points) 70

60

50

40

30

20

10

0

Time of measurement

Pre-intervention Post-intervention Six-month follow-up

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

....

...

.. ...

Notes: Scores are expressed as a percentage of the maximum score on the scale. Vertical lines represent the 95% confidence intervals of the mean.

The mean start score in this sub-group was 47.4 percentage points (95% CI: 32.9 to 62). This was comparable to the mean start score in the entire cohort (mean of 54.4, 95% CI: 46.8 to 61.9). The mean improvement in this group between pre-intervention and post-intervention was 27.6 percentage points (95% CI: 13.5 to 41.7). This was again similar to the pre-intervention to post-intervention improvement in the larger cohort, for which the mean improvement was 24.6 percentage points (95% CI: 15.6 to 33.5).

The average score at six months after treatment can be compared to the post-intervention scores in this group: the mean change in score in this six-month period was an improvement of 1.09 percentage points (95% CI: –9.1 to 11.3). Thus it appears from this sub-sample that the improvements seen at the end of the intervention were maintained in the cohort after six months, although there was some fluctuation of scores (as indicated by the wide confidence interval around the estimated change between treatment completion and six-month follow-up).


Chapter 10: Economic Analysis The analysis of the costs of delivering the PMHIs in this evaluation looks at the costs of service provision for a small number of initiatives against the outcomes yielded in terms of service-user numbers and mental health outcome scores (K10, PHQ-9). Economic data from the initiatives was collected at the end of July 2007, and the service-user data included in this analysis is derived from that end-of-July collection. Table 10.1 provides the ranges of costs of different types of service provision.

Table 10.1: Range of costs for different services offered by initiatives

Type of position/service Range ($NZ)

Counsellor (1 FTE) Project manager / non-clinical co-ordinator (1 FTE) Clinical/team leader GP visit Talking therapy contracted Clinical psychologist (1 FTE) Co-ordinator/nurse clinical (1 FTE) Medical adviser (0.1 FTE)

48,000–62,000 63,500–95,000 70,000–85,000 50–60 per visit; 170 flat fee 80–150 per session 70,000–75,000 60,000–85,000 8,000–24,000

Costs varied significantly across regions, such as costs being higher in Auckland compared with the South Island.

10.1 Analysis of positions created and services provided by initiatives

Talking therapy provision Talking therapy was provided by both contracted providers and staff employed by the PHO. Contracted providers were paid between $80 and $150 per session. Those with a counselling background tended to be paid the lower rate ($80–$100), with the higher rates paid to psychologists providing cognitive behavioural therapy (CBT). Employed talking therapists were either psychologists or counsellors.

GP visits Some initiatives paid GPs a flat rate, which they could use as they saw fit (eg, as four 30-minute visits or eight 15-minute visits), while some services paid GPs per visit. Some services paid the same rate regardless of whether it was a GP or nurse who provided the intervention.

Packages of care The term ‘packages of care’ can mean different things in different initiatives. For example, some initiatives include only one service type in their package of care, (eg, talking therapy provision) while others will include GP consultations and social support such as child care or transport.


The cost of standard packages of care ranged from $400 (covering talking therapies only) to $550 (three GP consultations, four CBT sessions). One programme included the costs for a package of up to $1,620, but this covered treatment for up to 18 months and could include up to 12 GP visits and six sessions of CBT. Some packages were priced differently if the person was on Care Plus (eg, GP visits were costed as less).

Medical adviser Two projects employed a medical adviser. These were both GPs with experience in mental health.

10.2 Findings on cost effectiveness Table 10.2 shows the set-up costs for each of the initiatives used in the cost-effectiveness analysis. Initiatives were asked to provide costing information on any of the following potential set-up costs: recruitment, implementation, IT, administrative overheads, and capital expenditure such as computer hardware or office fit-outs.

Table 10.2: Set-up costs for initiatives, including breakdown by IT and other costs

Programme Total set-up costs IT set-up costs Other set-up* costs ($NZ) (% of total) (% of total)

Initiative A $86,220 $27,140 (31.5) $59,080 (68.5) Initiative B $183,000 $127,500 (69.7) $55,500 (30.3) Initiative C $50,000 $8,000 (16) $42,000 (84) Initiative D $61,000 $18,500 (30.3) $42,500 (69.7)

* FTE for people setting up the project, as well as training and capital expenditure (eg, laptops, cellphones and office fit-out).

Table 10.3 summarises the cost-effectiveness findings for all of the analysed initiatives, as well as noting the mean pre-intervention score (on the percentage point scale) for each initiative. Note that this pre-intervention score was calculated for all service users with start scores, and was not restricted to those service users who had both start and end scores.

Table 10.3: Costing information and coverage of service users with both pre- and post-intervention scores

Initiative Total cost No. of Cost per Cost per percentage % with pre- Mean pre-service service point improvement and post- intervention score users user (95% CI)* intervention

scores† (95% CI)

A $409,384 445 $919.96 $36.96 (28.13–53.85) 5.2 58.21 (56.47–59.97) B $694,888 1063 $653.70 $18.91 (17.94–20.01) 61.2 72.31 (71.6–73.01) C $755,072 1295 $583.07 $20.85 (18.74–23.49) 14.8 49.53 (48.52–50.53) D $716,987 767 $934.79 $44.41 (40.23–49.56) 40.7 45.95 (44.98–46.92)

* One initiative used the PHQ-9 measure; the others used the K10. † Percentage of service users in initiative with both pre- and post-intervention scores.


The different groups in the PMHI workforce were paid a range of costs. GP visits were funded from $50 to $60 per visit, talking therapy sessions from $80 to $150 per visit and co-ordinator/ nurse roles received $60,000 to $85,000 per annum.

Set-up costs for the initiatives profiled ranged from $50,000 to $183,000 and covered items such as time spent formulating the response to the request for proposal, recruitment, implementation, IT, administrative overheads, and capital expenditure such as computer hardware or office fit-outs.

The total cost of service provision for the time period being studied ranged from $409,384 to $755,072, with the mean cost of service provision for the four initiatives being $644,082. The cost of service provision per service user ranged from $583.07 to $934.79. The estimated cost per percentage point improvement on the outcome measures ranged from $18.91 to $44.41.

There has been no previous economic analysis of primary or secondary mental health care interventions completed in New Zealand, and so it is difficult to provide a context for the findings of this analysis. The number of caveats relating to the collection of costs and outcome data, coupled with the low rate of return for post-intervention measures, mean that caution should be applied when drawing conclusions from the cost-effectiveness analysis (Knapp 1999). However, the information provided on the costs of units of service provision, salary ranges for positions and set-up costs can be read with some certainty.

A major point to note is that costs per service user are sensitive to economies of scale. The initiatives seeing fewer service users have significantly higher costs per user. Part of this effect is due to initiatives with larger numbers of service users being able to spread their set-up and administration costs across more users, in effect lowering the cost per service user. As a result, the costing per service user seen is highly dependent on factors such as initiative location and population density.

It is interesting to note that once the IT costs are removed from the set-up costs, the set-up costs for other factors are fairly similar, ranging from $42,000 to $59,000. Even with a large IT set-up cost (eg, in Initiative B), the cost per service user can still be low if enough service users are seen.

The variable cost of talking therapies can have a significant impact on the total cost of the service. For example, initiative A was able to access talking therapies for as little as $65 an hour, with the usual cost being $75–$85 per hour, while initiative D was having to pay $130 per hour.

Quantitative data on cost effectiveness should also be considered in conjunction with service-user and provider perceptions of the value of accessing such services. Many service users mentioned the implications of the costs to themselves and their families if this care had not been available.

No, I’d just like to say it’s really good to have that service available, and I wished services like that had’ve been around for a longer time, especially my period when I was more unwell, yeah. (Service user)


The economic implications are particularly significant if potentially serious mental disorder or suicidality is averted.

Then we got to this thing – I think it was only – the suicide thing was only a quick – a thing that your mind gets into, and it’s – as soon as you talk to someone, you think, how can I ever have thought that, because you can’t tell anyone else, and I wouldn’t have told him [the GP]. (Service user)

Further data on cost-effectiveness is available in the extended report.

10.3 Key points • Caution should be exercised when drawing conclusions regarding cost effectiveness by score

improvement, because of the use of different outcome measures in the different initiatives, the low rate of return of outcome measures for some initiatives, and the non-standardisation of reporting of costs.

• The costs of services and staff (FTEs) varied greatly across regions.

• All services will have a certain level of set-up and ongoing running costs, but the greater the number of service users seen, the more cost effective the service becomes.

• Smaller initiatives in locations with small local populations should not be regarded as less cost effective, due to the almost fixed nature of the set-up costs.

• Initiatives with the ability to meet some service users’ needs without spending money on a package of care may be more cost effective.


Chapter 11: Conclusions and Service Implications This chapter provides overall conclusions for the report and discusses the service implications.

11.1 Context Two particular themes provide the overall context within which the PMHIs should be evaluated. These are:

• the overall prevalence and nature of common mental health conditions and disorders that are seen in primary care

• the current workload and working practices of general practice and primary care in New Zealand.

As noted in the introduction to this report, New Zealand shares with other OECD countries a high prevalence of the three most commonly identified mental health conditions: anxiety, depression and substance-use disorder. The success or otherwise of these initiatives will thus depend partly on whether they address the needs of people with one or more of these conditions. From the data collected on service users, however, it is clear the many of them suffered from a combination of complex life stress and psychosocial problems, and hence responsive initiatives must be able to accommodate service users with these problems as well.

The second theme concerns current workload and work patterns within New Zealand general practice. In the complex competing demands of the GP consultation there is a tension involved in responding to mental health concerns that take increased time (Klinkman 1997). In New Zealand this is compounded by the part fee-for-service system, which means that from the patient’s perspective there are financial barriers to present to, and continue their engagement with, health professionals for mental health problems.

11.2 The overall effectiveness of the PMHIs The fact that 26 different initiatives were developed and sustained over a two-year period should already be seen as a measure of success. There was no previous consistent platform from which to provide organised primary mental health services, and there was a lack of resources for general practice and primary care teams to access. Given the multiple competing demands within primary care, it is extremely unlikely that primary mental health could have been prioritised without the additional investment in infrastructure.

Drawing on themes from the international literature, it is clear that a number of options can enhance the existing effort of individual GPs, primary care teams and general practices, including:

• changing the balance of GP and practice nurse routine activity, redistributing the work to add or take away time for provision of mental health care

• developing and implementing new primary mental health professional roles

• increasing access to other therapists and therapies

• improving liaison with other mental health services


• improving the presentation, detection and recognition of mental disorders as a problem

• improving the co-ordination of different disciplines (primary care, mental health, public health), and enhancing intersectoral collaboration between health social services and education.

The PMHIs successfully incorporated all of these themes within the various initiatives. The majority managed to increase the amount of time GPs were able to allocate for primary mental health, albeit to a relatively small number of service users. The high proportion of service users who received extended GP consultations represent not just an additional investment of the time known to be an important component of effective primary care (Howie et al 2000), but also changed triaging and referral patterns. The type of care delivered would not be sustainable using previous fee-for-service frameworks. New primary mental health professionals have been recruited, and their roles have been developed and implemented. These roles are seen as being very acceptable to existing primary health care professionals, and have extended beyond the care of service users into mentoring and education, and working in partnership with existing primary health care staff.

Another marker of success is the rapid evolution of service development. Many of the initiatives had little prior experience or infrastructure, yet within a two-year timeframe established viable clinical services with demonstrable successful clinical outcomes. This process has also involved testing new referral and management networks across new interdisciplinary teams. The process compares very favourably with other primary care initiatives such as the implementation of the CarePlus initiative (Ministry of Health 2006).

Overall, the PMHIs were able to demonstrate significant clinical improvements in those who were seen. The data shows that up to 80% of service users benefited from a variety of interventions offered to them. Taking into account expected spontaneous resolution rates, this represents a significant and beneficial treatment effect, and compares favourably with clinical outcomes seen in the Australian Initiative to Improve Better Mental Health Outcomes (Winefield and Turnbull 2007). Across these New Zealand initiatives the mean improvement was 11.7 points on the K10 scale (95% confidence interval = 8.9 to 14.7); in the Australian study, for the 49 individuals who had between three and five psychological interventions, the mean improvement was 12 points on the K10 scale (95% confidence interval = 9.5 to 14.5). Randomised controlled trials would be able to test the magnitude of this treatment effect with more certainty.

In line with the terms of reference of the request for proposals, initiatives generally prioritised service users from perceived high-needs groups, which included Māori, Pacific peoples and those on low income. Given the known prevalence of mental health conditions in these groups, and the difficulty they have in accessing services, these service users were able to clinically benefit more compared to their previous options under ‘usual care’. Relatively few service users would have been able to easily access and afford the extended consultations and psychological therapy provision made available to those in the initiatives.


The qualitative data revealed a widespread view that access to primary mental health care is difficult, and hence service users would not have fared as well without the initiatives. There is no indication from the data whether the overall treatment provision was inappropriate or ‘excessive’. Treatment protocols in general were in line with the current – although fairly limited – evidence base on practice for primary mental health care, in terms of pharmacological and psychological therapy as well as social or other support.

As we have seen, those who received care through the PMHIs achieved significant clinical benefits. The overall change in outcomes scores compares favourably with other published literature using those scales, and qualitatively service users were very positive about the clinical care they received. A further important theme regarding the impact on the life of service users can be drawn from the six-month follow-up data. In this sub-sample the improvements seen at the end of the intervention were maintained after six months.

11.3 Appropriateness of service provision The Ministry of Health’s intention was for the PMHIs to address the mental health needs of those with presumed mild to moderate disorder (an estimated 17% of the population). This target population was different to the 3% of those with severe and enduring disorder, who were to be managed by secondary care. From qualitative interview data from both service users and providers, and assessment of the entry and outcome assessment scores, the PMHIs appear to have addressed the needs of the defined population. Small numbers of service users were referred to secondary care outside the defined criteria (180 out of 13,000, or 1.5% – see the extended report).

Quantitatively, the mean pre-intervention scores indicate a moderate to moderate/severe level of acuity. There are three possible explanations for this, partly confirmed by interview.

• It is likely that the overall level of complexity and acuity within the community is greater than that perceived by planners and secondary care.

• Providers describe an environment in which referral to secondary care services is challenging.

• It is possible that there was a pool of known cases who were immediately referred to the newly developed services.

The service implications in terms of volume are considerable, as many of the initiatives reached their capacity threshold after approximately one year of operation, and many others have fixed a threshold for access that would equate more to moderate/severe rather than mild/moderate. It is therefore important that access to primary mental health care is debated openly so that clear decisions can be made regarding the threshold for care.

The complexity of this issue is compounded by the nature of the problems that are presented. Nearly a quarter of all those seen by the PMHIs did not fall into a clear-cut diagnostic category, but consisted of sub-threshold primary mental health conditions, or service users with significant psychosocial problems and socioeconomic hardship. Services for primary mental health care have the capability to assess the initial presentation of these problems and address undifferentiated and sub-threshold symptom complexes as well as well-defined disorders such as depression and anxiety.


11.4 Models of care A range of service delivery models was developed, offering choice to both service users and practitioners. All services were offered at no charge to services users. Every model contained some elements specific to local need. It is unlikely that any single model could be universally applied across all regions because none contained all the successful elements of an optimal model (see Figures E1 and E2.).

From this variety of model types we have endeavoured to define what could be regarded as a ‘generic’ optimal model, which is described in the executive summary. A number of themes are important in terms of service development. Contained within the initiatives and common to primary care itself is the notion of a planned pathway of care, in which more intensive intervention is available to those cases that are more severe, more complex, or fail to improve after initial, less intensive intervention. The degree to which the PMHIs provided a structured ‘stepped care’ model varied, but the evolution of existing initiatives and new developments must take this into account.

Finite resources as well as the existing evidence base favour this approach, with the implication being that mild symptoms will be managed with advice on self-management from the existing primary health care team, moderately severe symptoms will be managed by referral to generic talking therapies (with or without pharmacotherapy), and access to more specialised therapies and therapists will be reserved for a small number of people with more severe symptoms.

The evidence from this evaluation is that there was little difference between the outcomes of the different talking therapy approaches, and that any observed differences in the effectiveness of different model types were most likely due to local system advantages and disadvantages rather than the overall model type itself. There were clear advantages to models used in PHOs that had previous experience in primary mental health and that obtained support and encouragement from their DHB, and this is important in considering the next phases of PMHI development.

The present PMHIs do not sufficiently address the mental health needs of children and young people. There is a high prevalence of mental health problems in young people (NZGG 2008) and an evidence base that indicates that adult disorders have their onset in late childhood and adolescence (McGorry et al 2007). It would thus seem prudent in future to place greater emphasis on child and youth services for these reasons. The service implications are complex, given the impact of child and youth mental health on other sectors (education and social welfare), and policy work is required at both the national and local DHB level to provide better intersectoral working than at present.

There was also a lack of focus on the mental health needs of the elderly within the PMHIs. This raises important questions as to what degree an ‘age and stage focus’ should be explicit in future developments.


Within the initiatives framework a number of providers have been able to deliver new primary mental health services for Māori or build on existing ones. These ranged from full kaupapa Māori services, to encouraging Māori responsiveness in mainstream services. There are positive signs in terms of responsiveness to Māori, with some initiatives providing access to Māori in excess of the proportion within the enrolled population, and the same positive benefits in terms of clinical outcomes. Given the high prevalence of some primary mental health disorders among Māori, it is likely there is still under-utilisation of services for Māori, and there are challenges for some PHOs and DHBs in determining what will best meet the needs of Māori: mainstream intervention, kaupapa Māori intervention, or the ability to access both. It is important that Māori providers are given the opportunity to evaluate service provision, including promoting an evidence base for kaupapa Māori therapies where it is currently in the early stages of development.

The evaluation data indicates an under-utilisation by Pacific peoples in the PMHIs. There are also lessons from successful Pacific initiatives that have implications for future service development. These include the need to specifically include health promotion and de-stigmatisation features, and to recognise that Pacific peoples will have significant variation in health beliefs and service requirements. Given the changing demographic profile in parts of New Zealand, particularly Auckland, the lack of a specific focus on the mental health needs of people of Asian origin could also be reviewed.

11.5 Workforce The introduction of the PMHIs has considerable service implications for the development of the primary care workforce. New primary mental health staff have been introduced into an extended primary care team, there has been the opportunity for staff to move from secondary to primary care, and there has also been the opportunity to extend the roles of existing team members, such as GPs and practice nurses.

As stated previously in this report, a significant part of the success of the initiatives derives from the investment made in dedicated staff, such as the mental health co-ordinators/nurses. They require appropriate administrative support to do their job well and need a framework for professional development and supervision. Further work is required to define an optimal structured training or education programme that includes all disciplines.

The evaluation was effectively a snapshot in time, and viewed the role of the primary mental health co-ordinators at a particular stage in primary mental health development. It is likely that over time the requirements of these posts will change. If existing staff such as GPs and practice nurses were able to undertake more primary mental health work in clinical assessment and management, this would require an increase in those existing staff members.

At present there is under-utilisation of the potential and skills of practice nurses, although they are probably subject to the pressure of competing demands on their time more than anyone else in the primary care team. Coherent decisions and clear strategy should be outlined for how practice nurses can best be involved in primary mental health care work.


The PMHIs are an example of effective interdisciplinary teamwork. In the majority of initiatives care was provided by more than one discipline. The PMHIs presented an exciting opportunity to incorporate, in a systematic way, the skills of therapists such as counsellors and clinical psychologists, who previously have usually only been employed in a private and limited capacity.

We have described the many different therapeutic styles and therapies used by mental health professionals to achieve the same outcomes within these initiatives. We believe that further work is required to refine the core therapeutic elements within current interventions, and also believe this may be the prelude to a new style of brief primary care talking therapy for those with mild to moderate conditions. This is in line with current work reassessing the structure and effectiveness of existing therapies such as cognitive behavioural therapy (NZGG 2008).

Such moves imply there may be the opportunity for therapists to tailor their existing work more appropriately to these primary care service users. With a few exceptions, most PMHIs concentrated on the organisation of talking therapies. We believe that further work is required within primary mental health to define an optimal balance between psychological and pharmacological therapies. This has important service implications, since GPs would appropriately manage the greater part of drug treatment in mild to moderate conditions.

11.6 Information technology, clinical decision support and assessment tools

The initiatives made relatively little use of advanced IT platforms for administrative and clinical management or structured clinical decision support for mental health. More widespread use of appropriate decision support could enhance the skills of the existing primary care team and promote more consistent patterns of service delivery. There was relatively little reference made to the use of available technology support for practice management systems or therapy from the Internet. PMHIs would seem to be in an ideal position to assess ‘e-therapy’ if its use and evidence base become more widespread.

A feature of the evaluation was the encouragement of the use of clinical outcome scores. Although these screening tools were primarily intended to provide quantitative clinical outcome data, it is also hoped that more widespread use of various psychological scales and scores will enhance clinical practice. The scales used by the initiatives are not diagnostic, although it seemed that both service users and providers used them in this way and found them helpful in providing a global assessment of acuity.

Although these tools were initially developed for screening, over time clinicians have begun to use them as a useful monitor of improvement, and at an organisational level some PHOs utilised initial scores as a threshold for treatment access. The evidence base for the use of some of the scales in this way is not strong, although the results from the evaluation provide a very significant degree of face validity. The further use of assessment and diagnostic tools should be encouraged to refine clinical assessment and service-user engagement. This would mean a significant change in primary mental health care practice.


11.7 Limitations Given the non-experimental nature of the framework within which these initiatives were developed, the conclusions are based on an assessment of what was happening in primary care at the time of the evaluation. This report cannot provide clear indications of the relative efficacy of one particular initiative against another. This was not the purpose of the evaluation, and given the complex environment in which the initiatives developed it is extremely difficult to directly compare initiatives in different geographical areas. It was not within the evaluation brief to overtly examine the impact on service users of offering no-charge primary mental health care. The report does provide clear indications and conclusions regarding a direction for primary mental health care in New Zealand, and identifies issues that will facilitate or hinder that development.

11.8 Further research This evaluation has used both quantitative and qualitative data to provide a rich picture of the overall impact of the initiatives on primary mental health care. Although the findings are robust within an evaluative methodology, a number of important questions remain unanswered, and the following should be prioritised for further research:

• the barriers to GPs and PNs engaging in primary mental health care

• the essential elements of a brief intervention and what variations are acceptable

• the impact of different types of brief intervention therapy on common mental health conditions

• the place of e-therapy in the management of common mental health conditions

• the optimal role of the interdisciplinary primary health care team in primary mental health care, and their enthusiasm, aptitude and skill base for this work

• the optimal balance between pharmacological therapy and psychological therapy in the management of common mental health conditions

• the question of whether primary mental health care be viewed as a chronic condition

• the key elements of self-management or self-help in this context

• the degree to which social complexity contributes to the development and natural course of primary mental health conditions.


References Andrews G, Slade T. 2001. Interpreting scores on the Kessler Psychological Distress Scale (K10). Australian and New Zealand Journal of Public Health 25: 494–7.

Arroll B, Khin N, Kerse N. 2003. Screening for depression in primary care with two verbally asked questions: cross sectional study [see comment]. British Medical Journal 327(7424): 1144–6.

Barr V, Robinson S, Marin-Link B, et al. 2003. The expanded chronic care model. Hospital Quarterly 7(1): 73–82.

Bashir K, Blizard B, Bosanquet A, et al. 2000. The evaluation of a mental health facilitator in general practice: effects on recognition, management, and outcome of mental illness. British Journal of General Practice 50(457): 626–9.

Benzeval M, Judge K, Whitehead M. 1995. Tackling Inequalities in Health: An agenda for action. London: Kings Fund.

Brown MC. 1999. Policy-induced changes in Māori mortality patterns in the New Zealand economic reform period. Health Economics 8: 127–36.

Cohen J. 1988. Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

Collings S, MAGPIe Research Group. 2005b. Disability and the detection of mental disorder in primary care. Social Psychiatry and Psychiatric Epidemiology. URL: http://dx.doi.org/10.1007/s00127-005-0984-x

Counties Manukau District Health Board. 2006. Whanau Ora Plan 2006–2011. Auckland: Counties Manukau District Health Board.

Crabtree BF, Miller WL. 1999. Doing Qualitative Research. Thousand Oaks, CA: Sage.

Dew K, Dowell AC, McLeod D, et al. 2005. ‘This glorious twilight zone of uncertainty’: mental health consultations in general practice. Social Science and Medicine 61(6): 1189–200.

Dew K, Morgan S, Dowell A, et al. 2007. ‘It puts things out of your control’: fear of consequences as a barrier to patient disclosure of mental health issues to general practitioners. Sociology of Health and Illness 29(7): 1–16.

Dowell AC, Garrett S, Collings S, et al. 2008. Evaluation of the Primary Mental Health Initiatives: Extended report 2008. Wellington: University of Otago and Ministry of Health.

Falloon IR, Ng B, Bensemann C, et al. 1996. The role of general practitioners in mental health care: a survey of needs and problems. New Zealand Medical Journal 109(1015): 34–6.

Friedli K, King MB, Lloyd M, et al. 1997. Randomised controlled assessment of non-directive psychotherapy versus routine general-practitioner care [comment]. Lancet 350(9092): 1662–5.

Gantley M. 1999. A Beginner’s Guide to Undertaking a Thematic Analysis. London: Royal College of General Practitioners.

Gask L. 2005a. Is depression a chronic illness? For the motion. Chronic Illness 1: 101–6.

Gask L. 2005b. Role of specialists in common chronic diseases. British Medical Journal 330: 651–3.

Goldberg D, Huxley P. 1980. Mental Illness in the Community: The pathway to psychiatric care. London: Tavistock Publications.


http://dx.doi.org/10.1007/s00127-005-0984-x

Harris R, Bradburn M, Deeks J, et al. 2007. METAN: Stata module for fixed and random effects meta-analysis [computer program]. Statistical Software Components S456798, Boston College Department of Economics. Revised 29 October 2007.

Herd R, Richards D. 2007. Wahine Tupono: A kaupapa Maori intervention programme.

Hickie IB, Davenport TA, Naismith SL, et al. 2001. Treatment of common mental disorders in Australian general practice [comment]. Medical Journal of Australia 175(Suppl): S25–30.

Howie JG, Heaney DJ, Maxwell M, et al. 2000. Developing a ‘consultation quality index’ (CQI) for use in general practice. Family Practice 17: 455–61.

Katon W, Von Korff M, Lin E, et al. 2001. Rethinking practitioner roles in chronic illness: the specialist, primary care physician and the practice nurse. General Hospital Psychiatry 32: 138–44.

Kessler R, Mroczek D. 1994. Final Versions of Our Non-specific Psychological Distress Scale [memo dated 10/3/94]. Ann Arbor, MI: Survey Research Center of the Institute for Social Research, University of Michigan.

Kiro CA. 2000. Kimihia mo te Hauora Māori: Māori health policy and practice. Albany: Massey University.

Klinkman MS. 1997. Competing demands in psychosocial care: a model for the identification and treatment of depressive disorders in primary care. General Hospital Psychiatry 19: 98–111.

Klinkman MS. 2003. The role of algorithms in the detection and treatment of depression in primary care. Journal of Clinical Psychiatry 64(Suppl 2): S19–23.

Klinkman MS, Schwenk TL, Coyne JC, et al. 1997. Depression in primary care: more like asthma than appendicitis: the Michigan Depression Project. Canadian Journal of Psychiatry 42: 966–73.

Knapp MJ. 1999. Economic evaluation and mental health: sparse past ... fertile future? J Mental Health Policy Econ 2: 163–7.

Kroenke K, Spitzer RL, Williams JB. 2001. The PHQ-9: validity of a brief depression severity measure. Journal of General Internal Medicine 6(9): 606–13.

Kuper A, Reeves S, Levinson W. 2008. An introduction to reading and appraising qualitative research. British Medical Journal 337: a288 doi:10.1136/bmj.a288.

Liamputtong Rice P, Ezzy D. 1999. Qualitative Research Methods. Melbourne: Oxford University Press.

Lincoln Y, Guba E. 1985. Naturalistic Enquiry. Beverley Hills, CA: Sage.

MaGPIe Research Group. 2003. The nature and prevalence of psychological problems in New Zealand primary healthcare: a report on Mental Health and General Practice Investigation (MaGPIe). New Zealand Medical Journal 116(1171): U379.

MaGPIe Research Group. 2004. Frequency of consultations and general practitioner recognition of psychological symptoms. British Journal of General Practice 54: 838–42.

MaGPIe Research Group. 2005. General practitioners’ perceptions of barriers to their provision of mental healthcare: a report on the Mental Health and General Practice Investigation. New Zealand Medical Journal 118(1222): 1–8.

Mathieson F. Are we as good as we think we are? Self assessment versus other forms of assessment of competence in psychotherapy. Cognitive Behaviour Therapist, in press.


McGorry P, Purcell R, Hickie I, et al. 2007. Investing in youth mental health is a best buy. Medical Journal of Australia 187(7 Suppl): S5–7.

McKinlay E. 2002. Local Adaptation of Mental Health Guidelines: A report for the New Zealand Guidelines Group. Wellington: Wellington School of Medicine and Health Sciences.

Minister of Health. 2001. The Primary Health Care Strategy. Wellington: Ministry of Health.

Ministry of Health. 2006. Review of the Implementation of Care Plus. Wellington: Ministry of Health.

Morse JM (ed). 1997. Completing a Qualitative Project: Details and dialogue. Thousand Oaks, CA: Sage.

Moussavi S, Chatterji S, Verdes E, et al. 2007. Depression, chronic diseases, and decrements in health: results from the World Health Surveys. Lancet 370: 851–8.

Murray CJ, Lopez AD. 1997. Alternative projections of mortality and disability by cause 1990–2020: Global Burden of Disease Study. Lancet 349: 1498–504.

Myette T. 2008. Integrated management of depression: improving system quality and creating effective interfaces. Journal of Occupational and Environmental Medicine 50: 482–91.

National Health Committee. 2007. Meeting the Needs of People with Chronic Conditions. Wellington: National Health Committee.

National Institute for Health and Clinical Excellence (NICE). 2007. Management of Anxiety in Adults in Primary, Secondary and Community Care. London: NICE, 54.

NZGG (New Zealand Guidelines Group). 2008. Identification of Common Mental Disorders and Management of Depression in Primary Care. Wellington: New Zealand Guidelines Group.

Oakley Browne MA, Wells JE, Scott KM (eds). 2006. Te Rau Hinengaro: The New Zealand Mental Health Survey. Wellington: Ministry of Health.

Patton M. 1980. Qualitative Evaluation Methods. London: Sage.

Patton M. 2002. Qualitative Research and Evaluation Methods (3rd ed). Thousand Oaks, CA: Sage.

Pere RR. 1984. Te oranga o te whānau: the health of the family. Hui Whakaoranga: Māori Health Planning Workshop. Wellington: Department of Health.

Pignone MP, Gaynes BN, Rushton JL, et al. 2002. Screening for depression in adults: a summary of the evidence for the US Preventive Services Task Force. Annals of Internal Medicine 136(10): 765–76.

Pincus A, Pechura C, Elinson L, et al. 2001. Depression in primary care: linking clinical and systems strategies. General Hospital Psychiatry 23: 311–18.

Pincus H, Pechura C, Keyser D, et al. 2005. Depression in primary care: learning lessons in a national quality improvement programme. Administration and Policy in Mental Health and Mental Health Services Research 33: 3–15.

Pope C, Ziebland S, Mays N. 1999. Analysing Qualitative Data. London: BMJ Books.

Prince M, Patel V, Saxena A, et al. 2007. No health without mental health. Lancet 370(9590): 859–77.

Rodenburg H, Bos V, O’Malley C, et al. 2004. General practice care of enduring mental health problems: an evaluation of the Wellington Mental Health Liaison Service. New Zealand Medical Journal 117(1202): U1077.


Tantam D. 2006. Psychotherapy in the UK: results of a survey of registrants of the United Kingdom Council for Psychotherapy. European Journal of Psychotherapy, Counselling and Health 8(3): 321–42.

Thomas J, Jasper A, Rawlin M. 2006. Better outcomes in mental health care: a general practice perspective. Australian Health Review 30(2): 148–57.

Vagholkar S, Hare L, Hasan I, et al. 2006. Better access to psychology services in primary mental health care: an evaluation. Australian Health Review 30(2): 195–202.

Wagner E. 1998. Chronic disease management: what will it take to improve care for chronic illness? Effective Clinical Practice 1(1): 2–4.

Winefield H, Turnbull D. 2007. Evaluating a programme of psychological interventions in primary care: consumer distress, disability and service usage. Australian and New Zealand Journal of Public Health 31: 264–9.

WHO. 2002. Innovative Care for Chronic Conditions: Building blocks for action. Geneva: World Health Organization.

Zigmond AS, Snaith RP. 1983. The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinavica 67: 361–70.


2009

Evaluation of the Primary Mental Health Initiatives:Summary Report

–

Evaluation of the Primary M

ental Health Initiatives:Sum

mary Report

–

Documents

Evaluation of the Primary Mental Health Initiatives ... · and Economics Research Unit (AERU), Lincoln University • utilisation of data from HealthStat (section 8.6): Barry Gribben