Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
An evaluation of the implementation,
outcomes and opportunities of the
Care Capacity Demand
Management (CCDM) Programme
FINAL REPORT
13 January 2015
Report Authors: Dr Chris Hendry
Director
NZ Institute of Community Health Care
www.nzichc.org.nz
Laura Aileone Margaret Kyle
Health Workforce Consultant Clinician Researcher
This report is the property of the SSHW Governance Group.
It is confidential and not to be copied or distributed
without permission.
Final CCDM Evaluation Report (January 2015) 2
Acknowledgements
The authors would like to acknowledge the SSHW Governance Group, SSHW Unit Director
and staff, key informants who contributed to this report and, importantly, the DHB staff
who hosted the site visits, met with the evaluators and maintained contact to update
and clarify findings for this report. A list of key informants is available in Appendix 1.
Copyright
The copyright owner of this publication is the SSHW Governance Group. Permission to
distribute and/or reproduce material from this publication must be sought from the
Group via the SSHW Unit Director.
Disclaimer
The report authors have taken great care to ensure the information supplied within the
project timeframe is accurate. However, neither the Institute nor the contributors
involved accept responsibility for any errors or omissions. All responsibility for action based
on any information in this report rests with the reader. The authors accept no liability for
any loss or damage of whatever kind arising from reliance in whole, or in part, on the
contents of this report by any person, corporate or natural.
Glossary
BAU BAU Business As Usual
CaaG CaaG Capacity at a Glance
CCDM CCDM Care Capacity Demand
Management
CDS CDC Central Data Council
CDS CDS Core Data Set
CEO CEO Chief Executive Officer
CFO CFO Chief Financial Officer
CIO CIO Chief Information Officer
CNM CNM Charge Nurse Manager
COI COI Committee of Inquiry
COO COO Chief Operative Officer
DAH DAH Director of Allied Health
DHB DHB District Health Board
DON DON Director of Nursing
DOM DOM Director of Midwifery
EOI EOI Expression of Interest
FTE FTE Full Time Equivalent
GM GM General Manager
LDC HaaG Hospital at a Glance
IOC IOC Integrated Operations Centre
KPI Key Performance Indicator
LDC Local Data Council
MECA MECA Multi Employer Collective
Agreement
MERAS Midwifery Employee Representation
and Advisory Service
NHPPD NHPPD Nursing Hours Per Patient Day
NSO NSO Nursing Sensitive Outcomes
NZNO NZNO New Zealand Nurses Organisation
PMS Patient Management System
PSA PSA Public Service Association
RN RN Registered Nurse
RTC RTC Releasing Time to Care
SOR SOR Standard Operating Response
SSHW SSHW Safe Staffing Healthy Workplace
VIB VIB Variance Indicator Board
VRM VRM Variance Response Management
Providing clinician led research and consultancy services througout the country; Improving
Outcomes Through Community Health Research
New Zealand Institute of Community Health Care
15 Mansfield Avenue | PO Box 36 126 | Merivale | Christchurch 8146
T: 03 375 4200 Ext 7801 | DDI: 03 375 4101 | Mob: 021 655 355
Email: [email protected] or [email protected]
Visit: www.nzichc.org.nz
Final CCDM Evaluation Report (January 2015) 3
Table of Contents
Acknowledgements ..................................................................................................................... 2
Copyright ..................................................................................................................................... 2
Disclaimer .................................................................................................................................... 2
Glossary ...................................................................................................................................... 2
Table of Contents .......................................................................................................................... 3
Executive Summary....................................................................................................................... 5
Evaluation Aim ............................................................................................................................ 5
Evaluation Methodology .............................................................................................................. 6
CCDM Programme overview....................................................................................................... 6
Final Evaluation Findings ............................................................................................................ 7
Programme resources ............................................................................................................ 7
Programme outputs ................................................................................................................ 7
Programme outcomes ............................................................................................................. 8
Programme impact .................................................................................................................. 9
Recommendations .................................................................................................................... 10
1. Continue the CCDM programme ...................................................................................... 10
2. Maximise and formalise the use of the SSHW Unit. ......................................................... 10
3. Enhance the CCDM tools and processes. ........................................................................ 11
4. Focus on completing the current roll-out in hospital wards in participating DHBs............ 12
5. Develop support processes for those implementing change. ........................................... 13
Introduction .................................................................................................................................. 14
Aim of the Evaluation ................................................................................................................ 14
Methodology .............................................................................................................................. 15
Governance of the evaluation process ................................................................................. 16
Phase 1. Current state assessment and Interim Evaluation Report. .................................... 16
Phase 2. Ongoing programme monitoring and feedback ..................................................... 17
Phase 3. Impact assessment and final evaluation report ..................................................... 17
Background .................................................................................................................................. 18
The Safe Staffing Healthy Workplaces Unit .............................................................................. 18
Rationale for a Care Capacity Demand Management Programme .......................................... 20
Development of the CCDM Programme ................................................................................... 22
Engagement with Unions .......................................................................................................... 23
Advisory Groups ........................................................................................................................ 23
Process of CCDM Implementation ............................................................................................ 23
Relationship between CCDM and TrendCare® .................................................................... 24
CCDM Programme Overview ...................................................................................................... 26
Components of the CCDM Programme .................................................................................... 26
CCDM Tools .............................................................................................................................. 28
Final CCDM Evaluation Report (January 2015) 4
CCDM Programme Implementation .......................................................................................... 30
Evaluation Findings..................................................................................................................... 32
Planned Impact of the Programme ........................................................................................... 32
Resources Supporting the Programme ..................................................................................... 33
The Safe Staffing Healthy Workplaces Unit .......................................................................... 33
DHB resources required to implement and roll out the programme ..................................... 33
CCDM Interventions Deployed .................................................................................................. 35
Organisational engagement processes ................................................................................ 36
Programme implementation .................................................................................................. 37
Developing a system platform............................................................................................... 38
Getting the base right ............................................................................................................ 40
Implementation of Variance Response Management ........................................................... 43
CCDM Programme outputs ....................................................................................................... 50
CCDM Programme Outcomes .................................................................................................. 53
1. CCDM as a pioneering workforce methodology ............................................................... 54
2. CCDM as an enabler of cultural change ........................................................................... 54
3. Fostering and maintaining a greater level of DHB and union partnership ........................ 55
4. Achievement of variance reduction and workload smoothing .......................................... 56
5. Direct impact on NHPPD and associated financial impact ............................................... 62
6. Improved ability for a DHB to harness its acuity data ....................................................... 67
7. Further programme potential for growth and development .............................................. 68
Impact of CCDM ........................................................................................................................... 69
1. Patient Safety and Satisfaction ............................................................................................. 69
2. Supporting Staff Health and Wellbeing ................................................................................. 70
3. Maximising Organisational Efficiency .................................................................................... 71
Conclusion ................................................................................................................................... 74
Recommendations....................................................................................................................... 75
1. Continue the CCDM programme ...................................................................................... 75
2. Maximise and formalise the use of the SSHW Unit. ......................................................... 75
3. Enhance the CCDM tools and processes. ........................................................................ 76
4. Focus on completing the current roll-out in hospital wards in participating DHBs............ 76
5. Develop support processes for those implementing change. ........................................... 77
Appendix 1.Qualitative Feedback on the Programme ............................................................. 78
References ................................................................................................................................... 84
Final CCDM Evaluation Report (January 2015) 5
Executive Summary
This evaluation provides an overview, feedback and recommendations on the Care
Capacity Demand Management programme (CCDM). Currently 12 DHBs are in various
stages of the CCDM roll-out, but only 11 DHBs were included in this evaluation, as the
other DHB had just commenced the programme.
The CCDM programme was initiated in response to the 2006 Safe Staffing Healthy
Workplaces Committee of Inquiry Report. This report identified strategies to address
concerns that hospitals were inadequately staffed by nurses and midwives to meet the
increasing complexity of patients. The evaluation was commissioned by the Safe Staffing
Healthy Workplaces (SSHW) Governance Group.
The CCDM programme was designed as a whole of (hospital) system approach that
focuses on the provision of tools, processes and organisational support systems to
undertake 3 key functions:
Matching the workforce availability and skill mix to patient acuity in each ward
on the day.
Providing a suite of indicators that enable a ‘real time’ view of the patient, the
ward and the hospital in relation to workforce availability and patient acuity, in
order to identify any gap between demand and capacity.
Providing tools that enable variance in the predicted workforce availability, skill
mix and patient acuity to be managed safely and efficiently on the day, using
standard operating responses (SORs).
The CCDM programme provides a comprehensive infrastructure for a whole of hospital
approach to managing the nursing and midwifery workforce to better meet the needs of
patients, staff and the organisation as a whole. It enables critical analysis of historical
hospital staffing resource allocation, fully supported by both the DHB executive team
and unions.
The programme uses an internationally validated electronic patient acuity tool to assess
the pattern of staff required to meet patient demand in each specific ward 24/7. A
workload analysis tool and an FTE calculation tool (Mix and Match Parts 1 and 2) inform
roster re-engineering, including skill mix changes. Another suite of tools (Variable
Response Management) provides ongoing review of patient demand and an agreed
response to unexpected demand. In all, the programme incorporates a suite of 11 tools.
The DHBs that choose to implement the programme are supported at specific stages of
implementation by consultants attached to the SSHW Unit. In its current form,
implementation of the programme throughout a whole hospital is likely to take three to
four years.
Evaluation Aim
The overall aim of the evaluation was to provide reliable and meaningful evidence to
inform decisions and processes related to the future implementation of CCDM into DHBs.
To achieve this, the evaluators were to:
1. Document CCDM in its entirety, including the underlying logic of each component.
2. Document the implementation of CCDM and identify the nature of changes since its
adoption.
3. Assess the impact of CCDM on patient outcomes, staff satisfaction and
organisational functioning.
4. Summarise the differences in adoption of CCDM between DHBs, and assess the value
of the CCDM Programme as a whole and its individual interventions, with regard to
addressing the key objectives of the stakeholders.
Final CCDM Evaluation Report (January 2015) 6
5. Make recommendations on future implementation of CCDM into DHBs and on the
critical factors required to ensure sustainability of the approach (if the value
proposition is supported).
Evaluation Methodology
A Programme Logic Model framework was used to guide the evaluation process, starting
with a ‘current state’ report developed for the SSHW Governance Group in March 2014.
This Interim Evaluation Report presented findings from:
Key informant interviews
DHB site visits and workshops with the executive teams, management and nurses
and midwives
Analysis of documents and reports from the SSHW Unit
Review of literature and reports.
A set of indicators was also developed to monitor the on-going roll-out of the
programme for the duration of the evaluation.
The final report includes updated and additional information to supplement the Interim
Evaluation Report findings, through the development of case studies, financial and
workforce metrics, further key informant interviews and a final workshop with the SSHW
Governance Group to discuss, clarify and critique the findings.
CCDM Programme overview
The CCDM programme consists of a complex set of activities and tools designed to firstly
diagnose readiness of the DHB to implement the programme, then engage the
organisation in a 18 month to 2 year process in order to introduce the various processes
and tools and embed the programme to achieve the three aims; matching workforce
availability to patient acuity, identifying and managing gaps in capacity and demand,
and safely managing unpredicted demand. Ultimately, when fully implemented, the
programme is designed to provide a ‘real time’ whole of hospital view for all staff,
enabling more direct intervention at the ‘coal face’ to manage patient flow. The table
below summarises the key activities and tools within the programme.
Table 1 Components of the CCDM Programme for each DHB
CCDM programme of activities Description of the activity
Validated patient acuity system
implementation
Confirm a functioning, actualise electronic validated
patient acuity tool with high inter-rater reliability (IRR).
Reliable validated patient acuity data
Negotiations & submission of EOI DHB initiates request to SSHW GG to implement the
programme. Planning CCDM implementation
CCDM 3 hour start-up workshop Orientation to the programme
Discovery process
Assessment of readiness for DHB to roll out the
programme and actions as well as pre-conditions
required to expedite the process.
All of staff survey
Interviews
Project/activity stocktake
Validated Patient Acuity tool® audit
Final CCDM Evaluation Report (January 2015) 7
Recruitment of site coordinator
DHB resources to support the programme Establish Organisational CCDM Council
Programme plan agreed and Letter of
Agreement signed
Process and timeframes agreed.
Workforce analysis (Mix and Match Part
1) for one ward.
Analysis of skill mixes, rostering, workload and base
staffing. Recommends changes to meet service
demand needs and patterns 24/7 throughout the
year.
Baseline measurement
Ward led action plan on findings
Local data council established
Make agreed changes
FTE calculation (Mix and Match Part 2)
for one ward.
Analyses the validated patient acuity data Nursing
Hours Per Patient (HPPD) day patterns for 6- 12 months
to calculate appropriate FTE to meet service
demand.
FTE calculation
Report with recommendations
Make changes based on
recommendations
Variance Response Management
Tools designed to obtain and manage ‘whole of
hospital’ prediction and safe management of
unexpected variance of patient demand
Churchill exercise
Capacity at a Glance screen (CaaG)
Integrated Operations Centre
Variance Indicator (scoring) Boards
Standard Operating Responses
Reallocation Policy
Essential Care guidelines
Core Data Set established Agreed set of indicators to monitor and benchmark
the impact of the CCDM Programme activities.
Ability to resource according to base
plan
Evidence to resource staffing appropriately 24/7.
Final Evaluation Findings
Programme resources
There was agreement that the actual resources of time, HR and IT required at DHB level
to implement and establish CCDM had been underestimated. This was exacerbated by
the fact that the programme was initially in development for the ‘early adopters’. The
programme has since consolidated and there is now more clarity over requirements and
recommendations. The Interim Evaluation Report and this final evaluation report have
been designed to expedite the roll-out process, so that the DHBs have in place a
comprehensive programme for improving health outcomes for patients, providing a
quality work environment and making best use of health resources.
Programme outputs
Programme roll-out has been much slower than predicted, initially owing to its
developmental nature. Processes used to engage the organisation as a whole were
necessary but time-consuming, in many cases because the DHBs were not as prepared
as they had initially thought. These initial processes, including mapping of all other
current DHB projects, assessment of staff readiness, a validated patient acuity tool audit,
Final CCDM Evaluation Report (January 2015) 8
and the allocation of resources for the programme, provided DHBs with a unique view of
their organisational preparedness to undertake a system-wide approach to workforce
analysis and planning.
In total the CCDM programme had been introduced to 11 DHBs. Among these DHBs, 51
(60%) of their wards had undergone a workload analysis, and almost 40% had
undergone an FTE calculation analysis of their staff availability against patient demand.
In almost every case, application of these tools produced evidence that changes
needed to be made to skill mix, rosters and the model of care to better meet patient
demand. This approach to change differed from past efforts in that the diagnostics
(particularly the workload analysis) involved the nurses and midwives providing the data
and the DHB/Union partnership supporting the changes required, including, in some
cases, increased resources.
Programme outcomes
Figure 1 below summarises the key outcomes of the evaluation. However, quantifiable
evidence of the actual impact of the programme was difficult to obtain, leaving the
evaluation team more reliant on qualitative evidence gathered during site visits and
from key informant interviews. This feedback was consistently very positive.
Figure 1: Key Evaluation Outcomes Framework
Reliable, regularly updated and consistent hospital-wide use of a validated electronic
patient acuity tool, together with widespread placement and use of the ‘Capacity at a
Glance’ (CaaG) screens enable a real time whole of hospital view of not just the staffing
and patient numbers, but the workforce skill mix and patient acuity. Use of the Variance
Response Management (VRM) tools to cope with unexpected demand anywhere in the
system enabled real time management of the hospital nursing and midwifery workforce
by those doing the work, monitored by management.
The full extent of this process in action appears to have:
increased staffing flexibility and mobility
made sharing of resources open and transparent, and
created an environment of trust between workforce and management.
The CCDM model supports a smaller integrated operations centre, because most
workforce management activities occur at the ‘edges of the system’, in and between
wards.
Final CCDM Evaluation Report (January 2015) 9
Programme impact
Analysis of the impact of the programme was framed within the SSHW Committee of
Inquiry Report (2006) goals of safe staffing:
assuring patient safety and satisfaction,
supporting staff health and wellbeing, and
maximising organisational efficiency.
For a number of reasons, the findings relating to the patient safety and satisfaction
indicators could not be specifically attributed to the CCDM programme. These reasons
included the variable extent of the roll-out within DHBs, the concurrent roll-out of the
national quality focused programme, and the lack of early development and
application of the Core Data Set (CDS), which had been designed to measure safety
and quality indicators.
The only reliable indicator that provided results which could be directly attributed to the
CCDM programme was variance reduction. The evidence obtained demonstrated a
smoothing of variance after CCDM implementation in the majority of wards from which
data was available. From a patient safety perspective, the availability of the Variance
Response Management (VRM) system provides an agreed, standardised process to
proactively manage unexpected variance in patient demand. From the clinician’s
perspective, it allows for a standardised response to negative variation and potentially
unsafe working environments. From the organisation’s perspective, it also allows for a
smoothing of positive variation and associated cost savings in Nursing Hours Per Patient
Day (NHPPD).
In the absence of reporting against an agreed and standardised Core Data Set, the only
data available on staff health and wellbeing indicated a decline in perception of
inadequate staffing levels by the staff who completed the surveys. One survey indicated
an improvement in staff wellbeing post implementation. Reliable data which could be
attributed directly to CCDM was not available. However, the evaluator site meetings
with staff did provide evidence of an unexpected enthusiasm for the programme, given
the survey findings. Staff felt engaged in the process and were intensely interested in the
outcomes of the diagnostic tools as they related to their wards. They seemed to have
trust in the process. However, the lag between data gathering and recommendations,
muted interest in following up with suggested changes.
With regard to maximising organisational efficiency, the programme initially has more
focus in this area. A notable difference identified at sites where the CCDM Programme
had become more established was the reported increase in communication and
transparency at each level of the organisation, including unions and nurses in wards. It
seemed that the programme had initiated more networking both between and within
the various levels of service and management in hospitals.
It is acknowledged by the evaluators that the CCDM programme to date has been
nursing focused and, except for midwifery, has not included other disciplines in direct
application of the tools into practice. The allied health workforce was seen to be
engaged and contributed to Variance Response Management in participating DHBs;
however, to date they have not been able to utilise an acuity tool, and therefore are
unable to complete other components of the programme. Reference to a ‘whole of
organisation’ approach may seem to overstate the impact of CCDM. Yet nurses do
make up the bulk of a hospital workforce, and logically improvements for nurses are likely
to impact on other disciplines as a result of increased efficiencies.
The evaluators acknowledge the work currently being undertaken by the SSHW Unit with
other disciplines, particularly allied health, to develop CCDM tools that meet their needs
and integrate with the current tools. Progress has been made in a midwifery specific
adaptation to the programme, and an upgrade to TrendCare® will allow full roll-out of
this in the near future.
Final CCDM Evaluation Report (January 2015) 10
A number of discipline and service specific advisory groups are currently providing
advice to the SSHW Unit on adaptations to the tools to meet their specific needs. The
evaluators noted general enthusiasm from allied health and medicine in particular about
having access to CCDM tools modified for their use.
From an economic perspective, concern has been expressed in various forums that
CCDM could potentially increase the cost of staffing. What we have found is that CCDM
provides a comprehensive infrastructure for a hospital to start effectively managing its
nursing spend, which has not been done well in the past. It is difficult to compare the
‘before and after’ financials. Pre-CCDM budgets were not necessarily based on any
accurate data-based approach around true workforce needs to meet patient demand,
nor were they benchmarked nationally, as they are with the FTE calculation. If the ward
has completed the workload analysis (Mix & Match Part 1) and FTE calculation (Mix and
Match Part 2), then the post CCDM financials are likely to provide a more accurate
reflection of the actual workforce requirements. It is at this point that the hospital and
wards should work on efficiencies, as they will have up-to-date reliable evidence of the
impact/cost savings. The few findings to date on the budgetary implications of CCDM
seem to indicate that it is relatively cost neutral. The reduction in use of casual staff and
the flexing up of existing staff, together with roster re-engineering and increasing skill mix,
all contribute to balancing increases in the FTE required.
The intent of this report is to present the CCDM programme as it is currently functioning
within DHBs. The programme provides a standardised and validated process for
matching and responding to the fluctuating and, at times, unanticipated demand for
patient care with the required workforce 24/7. If the ward/hospital/DHB does not
continue to maintain the programme, monitor its performance and respond
appropriately to the patient care demand on the day and over time, it runs the risk of
being viewed by nurses with scepticism. The programme will be blamed for ‘not
working’, rather than the organisation(s) being blamed for not responding appropriately
to an obvious staffing deficit or surplus.
Recommendations
Following this 12 month evaluation, the evaluators recommend that the SSHW
Governance Group achieve a national commitment to rolling out the CCDM
Programme to all wards in all hospitals in New Zealand. The following recommendations
are made by the evaluators to modify and expedite the current processes.
1. Continue the CCDM programme
This programme provides a safe level playing field for front line hospital staff in the drive
to provide efficient and effective health services. The programme integrates well with
other quality initiatives. Fully implemented, it will enable national goal setting and
benchmarking.
DHB Chief Executives
1.1 All DHBs should implement the CCDM Programme.
2. Maximise and formalise the use of the SSHW Unit.
The SSHW Unit has a unique national overview of the functioning and potential of DHB
hospitals throughout the country. In rolling out the CCDM programme for DHBs, it
performs a vital function as a change agent. In order to maintain consistency, retain
highly skilled consultants and achieve efficiencies in programme roll-out, ongoing
development and benchmarking, the Unit needs to be retained on a permanent basis.
DHB Chief Executives
2.1 The SSHW Unit should become a permanent structure facilitating the programme
roll-out. Maximising the use of the expertise in the Unit will act to benefit the roll
out and further develop the programme in other service areas and disciplines.
Final CCDM Evaluation Report (January 2015) 11
2.2 The Unit should also facilitate national benchmarking activities and national
networking to support the change processes required.
2.3 The Unit should be resourced appropriately to undertake this role and achieve a
balance between development-focused work and support for current roll-outs.
Ideally a set of key performance indicators relating to the roll-out should be
developed for the SSHW Unit to report against.
SSHW Unit Director
2.4 Currently the SSHW Unit has a wealth of knowledge and experience in all facets
of the CCDM programme, with each consultant allocated a specific DHB.
Consideration should be given to the consultants specialising in components of
the programme and working collaboratively as an implementation team with all
DHBs.
2.5 It is recommended that the SSHW Unit, with its programme expertise, provide
centralised support and management of the workload analysis and FTE
calculation (Mix and Match Part 1 and Part 2) including analytical capacity, to
ensure a quick turnaround of reports.
Ministry of Health and DHB Chief Executives
2.6 Manage the negotiations of a national licence with the current validated patient
acuity tool provider, formally overview the management of the tool’s
developments (to prevent hybridisation and different versions being in use
throughout the country), and facilitate access to upgrades.
3. Enhance the CCDM tools and processes.
At this point, the CCDM tools and processes should be viewed as a complete
programme. As such, the focus now needs to go on refining the tools and ordering their
implementation, so as to achieve the most effective and efficient implementation and
ongoing maintenance. The power point presentations, reports and associated
documents currently present the programme in an exceptionally complex way, and
need to be simplified.
SSHW Unit Director
3.1 Streamline the CCDM initial resource for DHBs, including an outline of their pre
and post CCDM resource requirements, particularly HR and IT resources, as well
as realistic timeframes.
3.2 Simplify the terminology and presentation of the programme, including the
reports. For example, consistently change Mix and Match Part 1 to Workload
Analysis and Mix and Match Part 2 to FTE Calculation.
3.3 Standardise as many processes as possible, including the provision of templates
to guide governance and planning processes, including report turnaround times.
3.4 Reconsider the order of the implementation process. For example, the Mix and
Match Part 2 FTE calculation could be completed in all wards prior to the
workload analysis, which may be considered only as a diagnostic tool for a
specific ward or service if necessary.
3.5 Adapt some CCDM information, assessment and training activities to be used by
clinicians in an e-learning environment, such as the Churchill Exercise. This would
allow staff to participate at a later date, for example when orientating.
Final CCDM Evaluation Report (January 2015) 12
DHB Chief Executives, Ministry of Health and Unit Director
3.6 Develop agreement on the Core Data Set nationally, and incorporate processes
to obtain reliable and regular reporting on these indicators early in the CCDM
implementation process. This would provide the DHB with a reliable set of data
against which they could measure the impact and benefits of the programme as
it rolls out, including staff satisfaction.
DHB Chief Executives
3.7 Support and encourage the hospital-wide use of the Capacity at a Glance
(CaaG) screen. Its widespread availability in public places for staff (and patients)
to view at their convenience was identified as the public face of the CCDM
programme.
3.8 Standardise the variance response management tools. It seemed that a
significant amount of time was spent customising these, although this made very
little difference in the end. Some DHBs were seeking permission to share.
4. Focus on completing the current roll-out in hospital wards in participating DHBs.
There is a risk that the SSHW Unit staffing resource will become dissipated as the DHB
programme roll-outs increase. Also interest in the programme has been generated by
other disciplines and services exposed to the potential of CCDM for them (for example,
allied health, mental health and midterm forecasting), requiring additional involvement
of the Unit. The evaluation indicates that priority needs to go towards perfecting the
system for nursing and using the Core Data Set indicators to provide more conclusive
evidence of the direct impact of this programme towards achievement of the ‘triple
aim’ in health care. A full roll-out for nursing and midwifery (once the TrendCare®
upgrade has been completed) is likely to then enable fast tracking of adaptation and
roll-out for other disciplines. Completing the roll-out in currently participating DHBs should
take priority. Useful learnings and efficiencies are likely to be gained for other areas once
CCDM has been rolled out to all wards in currently participating DHBs.
SSHW Governance Group
4.1 Dedicate priority resource to completion of full CCDM roll-out for nurse and
midwife staffing in all currently involved DHBs.
SSHW Governance Group and SSHW Unit Director
4.2 Negotiate agreed deadlines for continued implementation with currently
involved DHBs.
4.3 Work with DHBs that have agreed to implementation to ensure that their
executive team and middle management maintain their support and
involvement in the programme.
DHB Chief Executives
4.4 Continue the internal resourcing of the CCDM programme during roll-out until it is
embedded within the organisation as business as usual.
Final CCDM Evaluation Report (January 2015) 13
5. Develop support processes for those implementing change.
One very clear barrier to CCDM implementation, maintenance and roll-out is the level of
comfort staff have with change management. The CCDM programme at ward level
generally requires a change of service delivery model, roster re-engineering and the
introduction of skill mix. Calculating the impact of these changes and planning and
implementing them effectively require nurse managers to have a significant level of
leadership and management skills.
DHB Chief Executives, Unions and DHBs
5.1 Provide change management training for staff prior to CCDM implementation.
5.2 Establish and foster support networks between those embarking on changes and
those which have successfully completed changes. For example, facilitate
networks with nurse managers in similar settings who are undertaking changes,
following workload analysis and FTE calculation.
Final CCDM Evaluation Report (January 2015) 14
Introduction
The Care Capacity Demand Management (CCDM) programme is a programme that
was developed in New Zealand incrementally from 2007 onwards, through a partnership
process involving District Health Boards (DHBs) and unions representing health workers,
facilitated by the Safe Staffing Health Workplaces (SSHW), Unit. Funded by DHBs, the
SSHW Unit is hosted in Central TAS (DHB shared services). The SSHW Governance Group,
made up of representatives from key stakeholder groups, including the Ministry of Health
(MoH), has supported, guided and advocated the programme’s development since its
inception.
The programme was designed in response to the findings of the Safe Staffing Healthy
Workplaces Committee of Inquiry (2006). This had been established to address concerns
that hospitals were inadequately staffed to meet the increasing complexity of patient
needs, therefore effectively making them unsafe workplaces for nurses and midwives.
This programme was designed as a whole of (hospital) system approach that focuses on
the provision of tools, processes and organisational support systems to undertake three
key functions:
Matching the workforce availability and skill mix to patient acuity in each ward
on the day.
Providing a suite of indicators that enable a real time view of the patient, the
ward and the hospital in relation to workforce availability and patient acuity, in
order to identify any gap between demand and capacity.
Providing tools that enable variance in the predicted workforce availability and
patient acuity to be managed safely and efficiently on the day using standard
operating responses (SORs).
The processes wrapped around achievement of these functions are described in more
detail elsewhere in this document. They play an integral role towards achieving a safer
workplace for staff and care venue for patients. Further, the CCDM Programme, coupled
with the use of the only currently available standardised and validated electronic
patient acuity tool, which is a vital prerequisite for the programme to function, has
enabled a level of trust in and transparency of workforce management at the bedside
which has not been previously experienced.
An initial pilot began in 2009, with three DHB hospitals involved in trialling the programme
and implementation. The programme has developed and continued to date, with 12 of
the 20 DHBs currently in varied stages of implementation. The CCDM Programme consists
of a set of interrelated activities and a suite of tools. It is estimated that these will take up
to three to four years per DHB to implement ward by ward.
This evaluation was commissioned in August 2013. It was designed to collect information
on the CCDM programme over a 12 month period, from October 2013 – September
2014.
Aim of the Evaluation
The overall aim of the evaluation was to provide reliable and meaningful evidence to
inform decisions and processes related to the future implementation of CCDM into DHBs.
To achieve this, the evaluators were to:
1. Document CCDM in its entirety, including the underlying logic of each component.
2. Document the implementation of CCDM and identify the nature of changes since its
adoption.
3. Assess the impact of CCDM on patient outcomes, staff satisfaction and
organisational functioning.
Final CCDM Evaluation Report (January 2015) 15
4. Summarise the differences in adoption of CCDM between DHBs, and assess the value
of the CCDM Programme as a whole and its individual interventions, with regard to
addressing the key objectives of the stakeholders.
5. Make recommendations on future implementation of CCDM into DHBs, and on the
critical factors required to ensure sustainability of the approach (if the value
proposition is supported).
Methodology
The evaluation covered 11 DHBs that were in varying stages of implementation. One
more DHB, which had only recently commenced implementation, was not included in
the evaluation.
In order to cope with the complexity of the evaluation, a Logic Model process1 was used
as a framework for the entire process. This allowed the evaluators to develop and use
standardised tools to measure the progress and impact of the programme over time in
all of these DHBs. Figure 2 below outlines the Logic Model framework within the context
of this evaluation.
Figure 2. CCDM programme evaluation: Logic Model framework
Using this framework, the specific problems that the programme was attempting to solve
were first explored and identified. Then two questions were asked: ‘If this/these are the
problem(s), what does the ideal situation look like (the vision)?’ and ‘What is the
evidence needed in order to know whether the vision has been achieved?” The
evaluation then had a beginning and an end point, with progress towards the
articulated vision systematically explored in relation to the resources, deliverables and
evidence of achievement. This process also allowed for changes in direction to be
identified and accounted for.
The CCDM programme operates at three levels: nationally through the work of the SSHW
Unit, at individual DHB level with the DHB CCDM Council providing oversight, and at
individual ward level. Using the Logic Model process, the national, DHB and ward level
processes were viewed as co-dependent. Success at one level was viewed as likely to
be dependent on activities in another, just as issues at one level could be caused by
interruptions at another. A separate focus on each of these levels allowed a more
systematic approach and identification of the inter-level dependencies.
1 http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html
Final CCDM Evaluation Report (January 2015) 16
Governance of the evaluation process
The SSHW Governance Group formed an Evaluation Advisory Group that included the
SSHW Unit Director, the MOH Chief Nurse, a DHB CEO, Director of Nursing (DON) and
representation from NZNO and PSA. This Advisory Group, with two co-chairs, provided a
touch point for the evaluators and communication with the Governance Group.
Using the Logic Model process as a framework for information collection and analysis,
the evaluation took place in three phases.
Phase 1. Current state assessment and Interim Evaluation Report.
Phase 1 was undertaken in four stages over a six month period, resulting in the
production of an Interim Evaluation Report in March 2014 for the SSHW Governance
Group.
The stages included:
1. A workshop with the SSHW Unit Director and consultants who were responsible for the
formal programme development and roll-out. This enabled us to test the logic model
evaluation methodology process (including the questions in Table 1 below), gain an
understanding of the components of the CCDM programme, and identify potential
evaluation informants and further sources of information. A working relationship with
the team was established to keep the evaluation informed of programme
developments that were being accelerated as the evaluation began.
2. Site visits or teleconferences with DHBs implementing CCDM. The eight site visits took
a day each and consisted of four focus groups with staff and additional key
informant interviews, each using the standard set of questions and requesting
evidence to support the responses. These visits also included viewing the control
centre (if there was one) and visiting wards that had implemented CCDM
components. On site focus groups were held separately with:
The executive team, in most cases including the CEO, COO and DON
The CCDM Data Council
Nurse, midwife and medical clinical leaders/managers
Union representatives, nurses, midwives, allied health staff and others.
Key informant interviews varied, but most were undertaken with the CCDM co-
ordinator, allied health staff, business analysts, duty management and medical staff.
The teleconference with one DHB followed a similar format to the site visit. Two other
DHBs were unable to participate in the evaluation because of other pressing issues.
3. Key informant interviews. Using the standard set of questions, a number of key
national and DHB informants were interviewed (see Table 1 below).
Table 2. Standard set of questions*
1. What is your involvement in, and knowledge of, the CCDM programme?
2. How would you describe the intent of the programme?
3. What is your experience / impression of the CCDM programme to date?
4. What do you think the impact of the programme has been to date?
5. Would you recommend the programme for further roll-out in your DHB and/or nationally?
6. Could you envisage this programme working as ‘business as usual’?
7. How would you describe enablers for programme implementation in the DHB?
8. What barriers to implementation do you see?
9. Can you comment on the role and function of the SSHW Unit and others assisting with CCDM?
10. Do you have any suggestions for how this programme could be improved?
11. Are you aware of an alternative model or methods?
12. Are there any processes that could expedite implementation to the best of your knowledge?
13. Any further comments on the impact of the programme from your perspective?
*These were used as a cue for the evaluators when interviewing and holding focus group meetings.
Final CCDM Evaluation Report (January 2015) 17
4. Completion of a document, record and literature review to contextualise the
emergence and development of the programme from inception. This was
incorporated into the Interim Evaluation Report with the findings from the other
activities in phase one.
5. Recommendations
The Interim Evaluation Report provided a series of recommendations around:
Articulating the programme as a whole
Managing barriers to the programme’s roll-out
Enabling a more rapid roll-out of the programme
Identifying indicators that could be used to measure the programme roll-out and
outcomes at ward, hospital and DHB level.
Phase 2. Ongoing programme monitoring and feedback
A template was developed to monitor and measure the progress of CCDM
implementation across the country. For each of the 11 DHBs, progress with each
component and phase of the programme was monitored. This data was collected and
collated monthly. The evaluation team were also updated monthly on changes and
further development of the programme.
Phase 3. Impact assessment and final evaluation report
This phase included the addition of the following evidence to inform the final report:
Case studies, developed to focus on specific aspects of the programme to
illustrate its impact
Financial and workforce metrics, obtained from the showcase wards of six DHBs,
pre and post CCDM implementation, to seek quantitative evidence of
programme outcomes and impact
Key informant interviews, to further examine the programme, including interviews
with those who do not currently plan to implement CCDM in their DHB
A workshop with representatives from the SSHW Governance Group and SSHW
Unit, to test the findings of the evaluation
Attendance at the CCDM Orientation workshop at Auckland Hospital, where the
programme was about to be rolled out
Production of the Final Evaluation Report, with recommendations on the future
development of the programme.
Informants contributing to this evaluation
There was a high level of support for this evaluation process in all sectors. In total, 255
individual informants contributed throughout this evaluation, some many times, to ensure
accuracy of interpretation, provide additional information, obtain data and verify
findings.
Table 2. Profile of all informants
Executive team Managers Programme
support
Nurses & others Unions
CEOs 6 Duty managers 6 DHB analysts 2 Educators 2 NZNO 37
DONs 12 Nurse
Managers
22 CCDM Co-
ordinators
10 Nurses 81 MERAS 2
CIOs, CFOs,
COOs
4 IOC Manager 4 TrendCare®
Co-ordinators
3 Midwives 4 PSA 5
DOM 2 Quality &
Safety
8 SSHW Unit
team
7 Allied Heath 12
GM 3 Service
Managers
4 SSHW
Governance
Group
7 National
Informants
7
DAH 2 Medical Staff 3
Totals 29 44 29 109 44
Final CCDM Evaluation Report (January 2015) 18
Background
A prime concern identified by the 2006 Joint Committee of Inquiry (COI) into Safe
Staffing and Healthy Workplaces was that growth in demand for health care was
projected to outstrip the capacity of the health workforce if it remained in its current
form. While most strategies to manage this problem focus on self-care, increased scopes
of practice, growth of family/whanau carers and the unregulated workforce (MOH,
2010;MOH, 2011), the COI’s response focused more specifically on the preservation and
sustainability of the current workforce, particularly within the hospital setting.
Nurses and midwives make up a significant proportion of the health workforce; therefore
the implementation of safe staffing and healthy workplaces was viewed as a critical first
step in preserving the current nursing and midwifery workforce and encouraging others
to join its ranks.
The SSHW Committee of Inquiry (COI) did an extensive literature search on safe staffing
and healthy workplaces, and noted the evidence of an association between contextual
factors, such as staffing levels, skills mix, and organisational design, and workload and
patient and staff outcomes, such as length of stay, patient incidents, mortality rates,
patient and nurse satisfaction and nursing retention rates (COI, 2006, Lankshear et al,
2005). This gave focus to the development of a national Safe Staffing Healthy
Workplaces Unit, for the purpose of exploring and implementing activities and processes
that could be used and/or developed to provide a safer and more sustainable
workplace for nurses, midwives and their clients/patients.
The COI (2006) report represented a commitment by the NZNO and DHBs to work
together to gain momentum on a mechanism for nurses, midwives and employers to
respond immediately if workloads exceeded the determined levels, and to achieve
sustainable solutions to safe staffing levels, developed in a way that had the confidence
of nurses and midwives. The report also identified seven interdependent elements to
achieve safe staffing and a healthy workplace. These included:
1. The requirement for nursing and midwifery care
2. The cultural environment
3. Creating and sustaining quality and safety
4. Authority and leadership in nursing and midwifery
5. Acquiring and using knowledge and skills
6. The wider team
7. The physical environment, technology, equipment and work design.
The recommendations from the COI report led to the development of the SSHW Unit in
2007 within the DHBNZ, with the task of implementing the recommendations into the
health sector.
The Safe Staffing Healthy Workplaces Unit
The SSHW agenda had its formal beginnings as an outcome of the 2005 MECA
negotiations between DHBs and NZNO. Agreement was reached between the parties to
collaborate on establishing a mutually acceptable, credible alternative to the nurse:
patient ratios proposed by NZNO. This agreement saw the establishment of the joint
national SSHW Committee of Inquiry (COI).
The recommendations of this group were jointly endorsed by the parties. They resulted in
the establishment of the Safe Staffing Healthy Workplaces (SSHW) Unit. An initial three
years of funding provided by the Ministry of Health (MoH) was followed by an additional
two years of joint funding by the DHBs and the MoH, and then a further two years of
funding by the 20 DHBs. The current funding expires at the end of June 2015.
Final CCDM Evaluation Report (January 2015) 19
The SSHW Unit was set up in 2006, at the request of the DHBs and the New Zealand Nurses
Organisation (NZNO), to support the implementation of the recommendations of the
2006 COI Report. With the advent of the National Terms of Settlement involving NZNO,
the Public Service Association (PSA) and the Service and Food Workers Union (SFWU),
and the expansion of the Unit’s work beyond nursing and midwifery, the scope and
governance of the Unit’s work have been revised over time to reflect this evolution. A
timeline summarising the development of the SSHW Unit is shown in Table 1.
The Care Capacity Demand Management (CCDM) programme was developed by the
SSHW unit to address key elements of the Safe Staffing Healthy Workplaces agenda, by
balancing the requirement to deliver quality patient outcomes in quality work
environments, in ways that make efficient use of the health resources.
Table 3. SSHW Unit development timeline
Year Activities
2005 MECA negotiations between DHBs and NZNO. Nurse: Patient ratios proposed by NZNO.
Agreement to jointly pursue a more sophisticated mechanism in preference to ratios.
2005-
2006
Joint national Safe Staffing Healthy Workplaces Committee of Inquiry
2006 Recommendations of the Committee of Inquiry presented to and endorsed by the
parties.
2006 Joint approach made to the Minister of Health to set up the SSHW Unit. Three years of
funding at $400k per year approved.
2007-
2009
SSHW Unit established with joint governance. Initial approach taken focusing on the
development of escalation plans in the 21 DHBs.
2009
Review of the escalation-based approach shows that deeper attention is required to
systemic issues around staffing methodology and variance responsiveness.
2009 Three DHBs recruited as national demonstration sites for a new approach. This saw the
emergence of the Care Capacity Demand Management (CCDM) Programme and the
adoption of patient acuity data as the foundation metric for the staffing methodology.
2010 An independent review undertaken, resulting in a decision by the parties to progressively
roll out the CCDM Programme to all DHBs
2011 Three further eligible2 DHBs became involved in the CCDM Programme. An additional
three DHBs acquired a patient acuity system. The PSA and SFWU became involved.
2011 A further two years of funding secured from the DHBs at $400k per year. Joint approach
to the Minister of Health resulted in a further $400k per year for two years to enable
acceleration of the implementation of CCDM. A commitment made to involve a total of
12 DHBs by the end of June 2013.
2012 Five further DHBs3 became involved in the CCDM Programme. Commencement of a
formal research programme to assess the impact. One further DHB acquired an acuity
system. Ongoing commitments to the SSHW agenda made by the parties in the NZNO
MECA.
2013 Three further DHBs became involved in the programme. Funding secured from all 20 DHBs
until June 2015.
The operational activities of the SSHW Unit are managed by the Unit Director with
Programme Consultants (5), who provide motivational, coordination, technical and
research based expertise and support, as well as facilitation of CCDM programme
implementation to DHBs. Figure 3 below outlines the organisation’s structure and related
groups.
2 Eligibility was based on using a validated electronic patient acuity programme, and commitment to a
partnership approach 3 Noting that the original commitment was to 12 of 21 DHBs participating. Following this, Otago and Southland
DHBs amalgamated. For the purposes of this process Southern is counted as 2 DHBs
Final CCDM Evaluation Report (January 2015) 20
Figure 3. Organisational representation of the SSHW Unit (2013)
Rationale for a Care Capacity Demand Management Programme
A review of literature and related e-material was undertaken to explore the evidence
base for the approach to the current CCDM programme and inform the evaluation
process. Since the COI reported in 2006, significant literature on SSHW activities has
become available online. We also searched DHB websites and www.google.com for
information, including previous evaluations posted online.
The SSHW Unit also provided a significant number of reports (including previous
evaluations), presentations and information sets relating to the CCDM Project. For
academic material, searches were made of the CINAHL, PubMed and Proquest
databases, with country restrictions to the UK, USA, Australia, New Zealand, Canada and
Ireland. Documents were also identified from the SSHW Unit, DHBs and NZNO. The HIIRC
secure website, started in 2013, holds all the programme resources, and DHBs which are
implementing the programme and Union staff have access to these. Overall websites
accessed included:
http://www.nzno.org.nz/home/campaigns/care_point
http://www.dhbsharedservices.health.nz/site/future_workforce/sshwu/overview/d
efault.aspx
http://ccdm.hiirc.org.nz/
There is consistent evidence that both the quality and the quantity of nursing care have
an impact on patient safety and patient outcomes (Aiken et al, 2002; Needleman et al,
2002; Duffield et al, 2010). The growth in literature on the relationship between nursing
staffing, workload, and the context of the work environment and patient outcomes has
reached a point where there are now systematic reviews (Dall et al, 2009; Kane et al,
2007) available to guide the programme. From the patient’s perspective, Kane et al
(2007) concluded that there was strong evidence outlining the positive effect of higher
RN staffing on patient outcomes. This is supported by a rigorous study by Needleman et
al (2011), which found that there was an association between patient mortality and
increased exposure to shifts that were deficient 8 hours or more below target staffing,
and that this increased risk was cumulative. An association between mortality and
nursing staffing was also found on wards that had a high staff turnover. The authors found
that these results justified flexible staffing practices that match staffing to need, as in an
acuity-based staffing model (Needleman et al, 2011).
SSHW Governance Group
Director SSHW
DHB Shared Services
Central Region’s Technical Advisory Service
(TAS) GM SSHW Unit Consultant
SSHW Unit Consultant
SSHW Unit Consultant
SSHW Unit Consultant
SSHW Unit Consultant
Participating District Health Boards
Ownership
Governance
Secretariat and
operations
Functional application
Community Services Advisory
Group
Allied Health Advisory
Group
Midwifery Advisory
Group
Mental Health Advisory
Group
Forecasting and Planning
Advisory Group
Final CCDM Evaluation Report (January 2015) 21
This study moves the question away from asking ‘whether staff turnover contributes to
patient mortality’, to asking ‘now what can we do about staff turnover?’
Lankshear et al (2005) reviewed 22 studies and found that a richer skill mix, especially of
registered nurses, and higher nurse staffing are associated with improved patient
outcomes. The literature also reports an association between high rates of nursing staff
turnover and adverse patient outcomes, as well as between inadequate staffing levels
and job dissatisfaction (McGillis Hall, 2005 and Gekinas and Bohlen, 2002 cited in COI,
2006).
The vision for the CCDM Programme is centred on achieving a quality work environment
and quality patient care with the best use of health resources. These activities are also
viewed as having a significant enabling role in achievement of the Institute for
Healthcare Improvement ‘Triple Aim’
http://www.ihi.org/Engage/Initiatives/TripleAim/Pages/default.aspx .
Figure 3. CCDMs triangle framework.
Source: SSHW Unit (2013)
With a focus on lean systems and the associated analysis of patient flow, a tremendous
amount of resources have been invested in moving the patient through the hospital,
particularly out of emergency departments (Scoville & Little, 2013). The wards are the last
stop before discharge, and nurses are reliant on a number of other activities and actions
being taken, often by others, to have the patient exit the ward. To achieve efficiencies,
efforts are made to minimise the per patient bed days; hence the plan from a
management perspective is to increase patient throughput, so as to utilise the beds
available. There is both predictable and unpredictable demand, from planned
(elective) and acute (emergency) admissions into DHB hospital wards. The increased
acuity of patients that has occurred as a result of early discharge to increase the patient
flow through the hospital has not always been matched with the appropriate nursing
levels required to manage acuity and turnover. This leads to a large increase in workload
for nurses that they have not been able to objectively measure, in order to argue for
increased resources. The nurse or midwife working in this environment of high churn and
heavy workload needs to have on hand tools to manage their workload as safely,
effectively and efficiently as possible.
A nurse who works an 8 hour shift in a 30–40 bed ward for the week will provide care
specifically for four to six patients with varying needs in hospital beds in at least two
different rooms, and will work as one of six to eight other nurses, many of whom will
require assistance with their patients too.
Final CCDM Evaluation Report (January 2015) 22
The day will be interspersed with changes in medication, medical examinations, off ward
diagnostic tests, changes in treatment orders, visits from inquiring family and
treatment/visits with allied health staff. The nurse will need to refer to written
orders/prescriptions/diagnostic results and convey changes in conditions/vital signs for
further treatment advice. In the meantime they will have to attend to the patient’s
personal care needs, which will be impacted on by their mobility. When the nurse goes
off shift at the end of the day, there will be at least two other 8 hour shifts carried out by
at least two other nurses; and when the nurse returns to work the following day, they will
have 15–20 minutes to brief themselves on the patients they are caring for, the staff they
will be working with, key activities taking place, the state of the rest of the ward and the
capacity of the rest of the hospital, particularly emergency.
Following two days off, there will have been eight shifts completed by others since the
nurse was last in the ward. The environment in which many of these nurses work is one
where traditionally:
The allocation of numbers of nurses per shift was historically calculated, such as 6 x 5
x 2.
The shifts were 8 hours, generally 0645-1530, 1430 – 2300 and 2245 – 0715.
There were mainly registered nurses, with few enrolled nurses or health care
assistants.
There was one ward clerk on five mornings per week.
Their model of care was generally based on Primary Nursing, rather than a team
based model.
Development of the CCDM Programme
Care Capacity Demand Management (CCDM) is described by the SSHW Unit as an
organisational approach to ensuring that the demand for patient care is matched
accurately and effectively with the resources required. Care Capacity is a term used to
define the total resource required by the patient/client in order to achieve an
acceptable outcome. This measure of capacity includes the staff, environment (plant
and tools), infrastructure and financial resources needed to produce that outcome.
Through application of a series of tools, the programme is designed to:
1. Obtain an organisational benchmark of staffing capacity and flexibility.
2. Enable the organisation at all levels to assess and redesign staffing resources to
better meet their needs.
3. Provide mechanisms to consistently and effectively assess and respond to
fluctuation in staffing needs.
4. Reduce the stress on the organisation and staff of reactive workforce management.
Four principles underpin the approach:
1. Optimising organisational resilience: the ability of the organisation to sustain
production and outcomes in varying situations.
2. Designing and operating the system in a way that stays within the boundaries of
maximum productive capacity.
3. Addressing the organisational ‘blind spot’ through providing sensitive and sentinel
data to inform decision making at all levels of the organisation.
4. A focus on studying and replicating success (doing more of what is working) rather
than studying and eliminating failure pathways (SSHW Unit, 2012).
Final CCDM Evaluation Report (January 2015) 23
The SSHW Unit determined that the strategies most likely to result in gain for an
organisation wanting to improve the match between capacity and demand are
anchored around three areas key challenges. These are outlined in Table 4.
Table 4. Components of CCDM (SSHW Unit, 2013)
CCDM component Description
Base staffing design Strengthening the ability of those involved in the base to
accurately forecast, plan, establish and reduce known or
predictable variance, using a mix and match process.
Intelligent
information
Improving the quality of the information that is generated from
the service delivery part of the organisation so that those
involved in the design of the base capacity know how the
system is actually functioning at the moment and over time and
can respond effectively.
Responding to the
unexpected
Improving the ability of those involved in service delivery to
respond effectively when variance occurs called variance
response management.
Engagement with Unions
A partnership was developed between the DHBs and NZNO, stemming from the COI
signalling that the relationship between NZNO and the DHBs had to change. Their
collaborative partnership engaged in the implementation of CCDM. This partnership was
a new way of working: each party’s agenda was put to the side in order to functionally
work together to achieve a safe staffing outcome.
Three demonstration DHB sites were selected for a pilot project, referred to as the 3D
Initiative. The evaluation of this initiative emphasized the strength of the Union/DHB
partnership. The evaluation emphasised the role that the DHB/NZNO partnership had
played in the progress that had been made. The partnership was found to have made a
significant culture change in improving trust between the DHB and NZNO. The basis for
this was that, rather than previous agendas being used, the data was trusted and was
used as a foundation for decision making. The workforce indicated that they were
confident to participate and engage.
In 2010 the health union partnership expanded, with the PSA engaging and MERAS
coming on board to develop a midwifery multi-party process.
Advisory Groups
The Advisory Groups have representatives from the relevant unions and the DHBS, and
are facilitated by the SSHW Unit. Overall, the advisory groups are tasked with delivering a
programme of work so that the patient acuity driven staffing methodology within the
CCDM programme is suitable and accessible to other clinical professions.
As the programme was developing, the methodology was extended from inpatient
general wards to explore its value in community health, mental health, allied health and
maternity. There are challenges however, for example, allied health does not currently
have the ability to extract patient acuity data for their clients, and the CCDM staffing
methodology, is based on patient acuity data.
Maternity had been slow to engage as a workforce with TrendCare because they
argued that it did not meet the needs of the New Zealand model of care (the vendor
has since updated the maternity component of TrendCare®, to be released in June
2015).
Process of CCDM Implementation
Initially, the Unit’s analysis and the tools developed focused on the need for escalation
planning. While considerable efforts were made with this strategy, a sector evaluation
undertaken by the Unit in November 2009 noted:
Final CCDM Evaluation Report (January 2015) 24
‘There appears to be a significant gap between the level of reported activity and
reported systems capability and the achievement of desired outcomes…. suggesting
that the sector is at high risk of under adaptation failure in relation to safe staffing and
healthy workplaces.’ 4
This lack of satisfaction with progress by all parties (NZNO, DHBs and SSHW unit) led to the
formation of a new strategy to develop new tools and processes to be trialled, refined
and demonstrated in particular settings. As noted above, three demonstration sites were
selected for a nine-month (subsequently extended to a year) pilot project, referred to as
the 3D Initiative. The Unit and the Partnership Resource Centre (PRC)5 became the
external support agencies working alongside the three DHBs involved (Counties
Manukau, Bay of Plenty and West Coast) to develop and trial what would later become
a suite of tools within the CCDM Programme6.
The SSHW Unit now uses an agreed set of tools, outlined below, to implement, imbed and
maintain the CCDM programme in DHBs. Each DHB is tasked with establishing
mechanisms and processes to ensure that the system remains functional and that new
staff are orientated to the programme.
The SSHW unit was designed as a temporary unit with an emancipatory model. Therefore
over time, the level of support to the DHB from the Unit lessens as the programme
become increasingly ‘business as usual’ for the DHB.
Up to the point of this evaluation, the CCDM in its entirety was developmental. Some
DHBs viewed it primarily as a series of tools, which had been implemented to varying
degrees by varying DHBs.
The programme should be seen as more than a set of tools. When implemented to its full
extent, its 3 areas of intervention and 11 tools together comprise a sophisticated
interlinked system that addresses the multi-layered complexity of healthcare contexts.
CCDM does appear to be now at a tipping point, and moving away from a
developmental programme to a period of consolidation and validation.
Relationship between CCDM and TrendCare®
In the early stages of developing the CCDM programme, the SSHW Unit Director Jane
Lawless and Consultant Maree Jury attended a TrendCare® Co-ordinators Workshop run
by the Director of TrendCare®, to gain an understanding of TrendCare® capabilities. In
late 2009, Cherrie Lowe went on to conduct Safe Staffing CMDHB Master Classes at the
request of Jane Lawless at BOP and Counties Manukau DHB. These Master Classes
played an important role in initiating the development of some of the Mix and Match
tools subsequently used in the CCDM programme.
TrendCare® provided the following information and documents to the SSHW, which
supported/support the development of the CCDM Programme:
1. A process and work study data collection form used to identify the type of work
conducted in a ward / department so that the correct skill mix can be calculated
(this was developed into the work analysis data collection sheet) and a timing study
methodology.
2. A work intensity profile diagram showing peaks and troughs in workloads for each
shift. (This was developed into the variance response testing in the FTE calculation)
3. Documents on skill mix requirements - recommendations made as a result of work
studies conducted in Australia and New Zealand.
4. A process to calculate the productive hours for one year for one FTE.
4 SSHW Unit (2009) 5 No longer functioning resource from the Department of Labour. 6 SSHW Supplementary Report on the 3D initiative, Innovations & Systems. (2010)
Final CCDM Evaluation Report (January 2015) 25
5. A process on how to determine FTE for a ward, considering HPPD required for care,
existing staff, staff turnover, graduates, etc. (The process was extended with the FTE
Calculation tool.)
6. A roster re-engineering spreadsheet with formulas. (Used as the bases for Mix and
Match Part 2 FTE calculation)*
7. A ward minimum staffing profile – recommendations for specific ward types.
8. A process for calculating the adjusted variance by considering patient throughput /
churn.
9. A process for planning / calculating manpower for a new ward / department.
10. TrendCare®’s acuity benchmark ranges for each patient type. (Used for FTE
Calculations)
11. A process for development of short term staffing plans.
12. TrendCare® implementation and training resources including Terms of Reference for
TrendCare® Steering Committee, an Inter-rater Reliability Testing Tool, Clinical
Training Booklets and a copy of the TrendCare® software program and all
associated documentation7
In July 2012, TrendCare® half funded a full time National TrendCare® co-ordinator role
within the SSHW Unit. The funding from TrendCare® ceased 12 months later and the role
was changed to Programme Consultant within the SSHW Unit. However, the consultant
continues to work reasonably closely with TrendCare® and remains the TrendCare®
expert within the SSHW Unit. The SSHW Unit continue to have access to many of
TrendCare®’s resources for education purposes. The benefits in this relationship are
attained through achieving national consistency around the way the tool is used and
understood.
There is a good level of interdependency between CCDM and TrendCare® (more fully
explained in the CCDM implementation section of this report). There would be definite
advantages for variance response management (VRM) and the Core Data Set (CDS) if
TrendCare® was used to its full potential, as it would allow for consistent reporting. In
Figure 4 below, CCDM activities are identified in red, but are reliant on reliable and
validated information obtained from TrendCare®. It is acknowledged that VRM also
relies on the Variance Indicator Board and VRM responsiveness, discussed later as part of
CCDM. It is also acknowledged that system performance management and dynamic
redesign require multiple points of data to formulate the entire big picture.
Figure 4. Interdependencies between TrendCare® and the CCDM Programme
7E-mail communication with Cherrie Lowe 17/12/13. Information confirmed by SSHWU 08/01/14
Validated Acuity Tool
(TrendCare®)
Responding to the
unexpected
(Variance Response
Management)
Tailored base staffing
design for each area
(FTE Calculations & Workload
Analysis)
Intelligent
information (Core Data Set)
(Core data set)
Functional partnership between unions and DHB
Final CCDM Evaluation Report (January 2015) 26
CCDM Programme Overview
An overview of the CCDM programme is provided below, including a table (Table 3)
presenting the ideal order of implementation. This complex programme has a number of
component parts, many of which can be, and in some cases are, used independently of
each other. The programme also has synergies with other programmes and initiatives
currently deployed in hospitals, with the intention of achieving more efficient and
effective health care delivery and improving the patient journey.
Components of the CCDM Programme
CCDM consists of a complex set of tools and activities designed to enable healthcare
organisations to align their human resources so as to better manage variability in the
demand for patient care. The key goal is to ensure that the service knows the hours of
care that need to be planned into the staffing design, the skill mix that will best meet the
care needs, and the time at which the care is likely to be required. This is addressed
through use of two tools, the Mix and Match Part 1 workload analysis, and the Mix and
Match Part 2 FTE calculation. CCDM also supports a service to develop a set of markers
and indicators that will alert the service to any gap between demand and capacity,
which is underpinned by standard operating responses (SORs).
Management of this gap in the capacity care demand match is addressed by the
Variance Response Management (VRM) suite of tools, which enable the DHB to better
forecast and manage unexpected gaps to maintain a safe environment for care. Thus,
CCDM was designed to facilitate DHBs’ use of an evidence-based methodology to staff
safely, improve patient outcomes, and build organisational resilience, as well as manage
resources effectively.
All DHBs contribute to the funding of the operational activities of the SSHW Unit that is
charged with developing and supporting the roll out of CCDM as a national programme
to achieve the recommendations of the SSHW Committee of Inquiry (2006).
All DHBs have the option of implementing the programme; however, implementation is
critically dependent on the use of the software product called TrendCare®, which is
currently the only validated patient acuity tool available. This product, used in a number
of countries including New Zealand, Australia and South East Asia, is an electronic
patient acuity, workload management and planning tool. Patient acuity is measured in
Nursing Hours per Patient Day (NHPPD), based on benchmarked HPPD related to over
150 ‘patient types’ that can be coded to DRGs. While the nurse on the day has the
ability to ‘actualise’ specific patient hours required, based on additional acuity factors,
TrendCare® is a validated acuity tool, and the more standardised the HPPDs become,
the fewer local changes are required.
Currently 16 of the 20 DHBs in New Zealand use TrendCare® in their hospitals, and 12 of
these DHBs are in various stages of the CCDM roll out. (Only 11 of these DHBs were
included in this evaluation, as the 12th DHB had just commenced the programme during
the latter stages of this evaluation.)
Table 5 summarises the components, tools and activities involved in the CCDM process,
and the time-frame estimated to roll the programme out ward by ward and hospital-
wide. This table is based on information available from the SSHW Unit and provided to
DHBs to inform their decision making and planning for a CCDM implementation.
Final CCDM Evaluation Report (January 2015) 27
Table 5 Components of the CCDM Programme for each DHB CCDM programme of activities
and tools
Description of the
activity
Standard
or custom
tools
Responsible
agent
Phasing
& timing
Validated patient acuity tool
implementation
Confirm a functioning
and actualise electronic
validated patient acuity
tool with high inter-rater
reliability (IRR).
S DHB
First 6 - 12
months Reliable validated patient acuity data
S DHB
Negotiations & submission of EOI
DHB initiates request to
SSHW GG
S DHB/Union
Planning CCDM implementation C SSHW
CCDM 3 hour start-up workshop Orientation to the
programme C SSHW
Within first
6 – 12
months
Discovery process
Assessment of readiness
for DHB to roll out the
programme and actions
as well as pre-conditions
required to expedite the
process.
S SSHW
All of staff survey S SSHW
Interviews S SSHW
Project/activity stocktake S SSHW
TrendCare® audit S SSHW
Recruitment of site coordinator DHB resources to support
the programme C DHB
Month 10
- 12
Establish Organisational CCDM Council C DHB/Union
Programme plan agreed and Letter of
Agreement signed
Process and timeframes
agreed. S/C
Workforce analysis (Mix and Match
Part 1) for one ward.
Analysis of skill mixes,
rostering, workload and
base staffing.
Recommends changes
to meet service demand
needs and patterns 24/7
throughout the year.
S ALL
3 month
period for
each
ward
Baseline measurement S ALL
Ward led action plan on findings C DHB
Local data council established C DHB/SSHW
Make agreed changes
C DHB/UNION
6 – 12
months
FTE calculation (Mix and Match Part 2)
for one ward.
Analyses validated
patient acuity Nursing
Hours Per Patient (HPPD)
day patterns for 6- 12
months to calculate
appropriate FTE to meet
service demand.
S DHB/UNION
4 months
for each
ward FTE calculation
S DHB
Report with recommendations S DHB
Make changes based on
recommendations C DHB
Variance Response Management
Tools designed to obtain
and manage ‘whole of
hospital’ prediction and
safe management of
unexpected variance of
patient demand
C DHB
From
months 6-
12
Churchill exercise S SSHW
Capacity at a Glance screen C DHB
Integrated Operations Centre C DHB
Variance Indicator Scoring C DHB
Standard Operating Response C DHB
Reallocation Policy C DHB
Essential care guidelines C DHB
Core Data Set Agreed indicators to
monitor and benchmark
the impact of the CCDM
Programme activities. S DHB/SSHW
6 – 12
months
Ability to resource according to base
plan
Evidence to resource
staffing appropriately
24/7. C DHB
M&M
part 2
Final CCDM Evaluation Report (January 2015) 28
CCDM Tools
One of the tasks that came out of the COI was to develop and deliver a staffing
methodology that included ‘best practice guidelines for patient forecasting and patient
workload management systems’ and a ‘tool kit’ (SSHW, 2006, cited in SSHW, 2013).
Acuity based rostering
There are controversies over which is more reliable for safe staffing: a system based on
nurse/patient ratios, or a system based on a patient acuity tool. Examples of mandated
nurse-patient ratios are found in Victoria, Australia and California. However, a systematic
review by Lane et al (2004) reviewed legislatively imposed minimum nurse staffing ratios,
and concluded that there was no support for minimum nurse-patient ratios for acute
care hospitals. This was particularly evident when skill-mix and case mix were not
considered.
In New Zealand, in many instances, nurse managers currently decide the number and
mix of staffing needed to optimize safe patient care, based on historical numbers per
shift balanced with their budgetary constraints. Internationally, ratio practices used
include informal ratios which are established by precedent, formal ratios set by hospital
policies, mandated ratios established by legal policy, and acuity-based ratios which are
flexible according to changes in patient acuity (Plummer, 2005). Mandated ratios have
the benefit of requiring little consultation, no technology, no costly tools, and no
maintenance or training; although they can be appealing, they provide little incentive to
account for nursing workload empirically. An argument for using an acuity-based ratio is
that nurses need to collect data around their work and interface with care systems as
part of the multi-disciplinary team; equally it is important for managers to have data on
resources and costings to support the arguments for fair and equitable workloads
(Plummer, 2005).
Acuity tools have come to the forefront for those providing health services as inpatient
turnover and complexity have increased. There is little evidence on the extent to which
acuity tools are being used, and only a few studies examine trends in patient acuity.
These do, however, confirm an increase in acuity. For example, a Canadian study
examined case-mix data for all acute care hospitals in Ontario from 1997 to 2002; it
found that the most complex patients increased by 144% and the least complex patients
decreased by 24% (Preya, 2004). There are few studies that have been designed to
examine patient acuity in relation to patient outcomes.
Different measurement acuity tools are available internationally, but there is a lack of
consistency in the literature about the definition of acuity and how it is measured, and
very few tools are validated (Brennan and Daly, 2009). TrendCare® is recommended by
the New Zealand Nurses Organization (NZNO) and SSHW Unit because it is a validated
tool, in that it uses objective data relating to the patient’s condition and studies the time
it takes to perform nursing cares.
Twigg et al (2013) concluded in recent research that the staffing method of Nursing
Hours Per Patient Day (NHPPD) is a cost effective initiative, as the investment of increased
nursing hours via the NHPPD staffing method has clinical benefits and cost savings for
improved nursing sensitive outcomes (NSO). This study recommended further research
into the economic benefits of nursing staff changes at a ward level to better estimate
the cost specific NSO. While there was no other available evidence examining the
economic impact of increased nursing hours in the NHPPD staffing method, Kalisch et al
(2011), in a cross-sectional research study, identified that increased NHPPD resulted in
lower levels of missed care. The study found that when there were lower NHPPD, there
were more episodes of care rationing, which affects patient outcomes.
In order to develop acuity based rostering, three key tool sets have been developed
and evolved over the past 5 years and form part of the CCDM programme. These are:
Mix and Match, a staffing methodology providing two tools, workload analysis and FTE
calculation, to tailor staffing design for each area/service.
Final CCDM Evaluation Report (January 2015) 29
Variance Response Management (VRM), a suite of tools to monitor and guide response
to unexpected workforce demand.
Core Data Set (CDS), a system designed to monitor and benchmark data produced by
the organisation that are sensitive to the impact of variance in capacity demand
and workforce response.
Mix and Match tools
The Mix and Match methodology uses two tools: Part 1 workload analysis, and Part 2 FTE
calculation. This methodology is part of the CCDM programme and innovative to New
Zealand, therefore no international literature exists to measure whether wards using it
perform better in terms of patient and staff outcomes than wards that have not
implemented it. The SSHW Unit recently completed its own Mix and Match Staffing
Methodology Evaluation (2013) to assess whether the methodology could accurately
determine nursing staff requirements for each shift. The evaluation concluded that shifts
designed to meet demand were working significantly better than routine non-designed
shifts, as staff perceptions of the work environment, care provided and reports of care
rationing had all improved (SSHW Unit, 2013). This positive finding may also have resulted
from the increased awareness by the nurses of the routine work flows and capacity
demand fluctuations, which are highlighted during the Mix and Match process. Patient
perceptions in the evaluation were equivocal.
It was interesting to note, however, that even when wards had implemented the
methodology, they still struggled at times, to match demand with capacity when there
were sudden increases in staffing requirements. The possible implication is that a staffing
base cannot be made too lean and still expect to be able to cope with a significant
unanticipated change in variance between demand and capacity. Furthermore, it is
noted that none of the participating wards had adopted all of the recommendations
from the ‘Mix and Match’ methodology, so perhaps further inquiries into this area are
required. What was clear was that shifts staffed below the recommended levels reported
increased care rationing, staff dissatisfaction and anxiety.
A 2012 study examining the cost of turnover in New Zealand supported the Mix and
Match methodology’s philosophical underpinnings: that if a ward is staffed to
recommended levels, it can be cost effective (North et al, 2012). This study identified that
wards with under-budgeted FTEs had a higher staff turnover and higher sick leave. The
authors argued that for the cost of every two nurses who turnover, one additional nurse
could have been employed, which may have prevented the turnover in the first
instance.
Variance Response Management tools
The SSHW Unit was also charged, as an outcome of the COI into safe staffing and health
workplaces, with developing and implementing ‘a toolkit of best practice nursing and
midwifery staffing systems and management of these systems’ (SSHW, 2006). Seven tools
were developed, which together make up the Variance Response Management (VRM)
system. As this suite of tools is also part of the CCDM programme and innovative to New
Zealand, no international literature exists to evaluate it.
The SSHW unit undertook a quantitative evaluation of the Variance Indicator Board (VIB),
a component of the VRM tool set (SSHW, 2013). The aim was to determine the level of
agreement between what the ward level reported on the VIB and the nurse’s individual
scoring of each shift. Almost half (49%) of the time, the individual nurse’s score for the shift
was different from the VIB board, and 80% of the changes which altered the clinical
status on the Board were not reported to the shift leader. As a result of that evaluation, It
was recommended that additional training for staff should occur, to help them
understand the roles they play in operationalising the VIB system more effectively.
There is no literature available to underpin and therefore critique the rationale for choice
of process of implementation and roll-out for the CCDM programme.
Final CCDM Evaluation Report (January 2015) 30
The Core Data set
The purpose of the Core Data Set (CDS) is to establish an agreed set of indicators from
the clinical floor to the Board. These indicators are performance indicators informing all
levels of the organisation about the demand capacity performance of the DHB. The
SSHW Unit has identified an evidence-based minimum core data set that measures
demand capacity matching performance. They are identified as the ‘safe 6’ which
relate to:
1. Clinical hours required versus clinical hours provided - are patients receiving all
the care they need?
2. Health and Quality Standard markers - are adverse events occurring?
3. Productivity - is the budget being maintained?
4. Flow - are flows and volumes being achieved?
5. Staff satisfaction - are staff satisfied with what they are able to achieve?
6. Work effort – is the work effort to maintain service levels reasonable?
Some of these metrics will already be collected by the organisation (such as harm
markers, and indicators related to resourcing). Those that are not are progressively
introduced as they are developed and tested. Additional indicators valued by the
organisation are also added to the ore Data Set. The SSHW consultant supports the
Central CCDM Council through a process of review of the recommended indicators and
a determination of their current status within the organisation.
CCDM Programme Implementation
The implementation of the CCDM programme is dependent on the DHBs not only
investing in the validated acuity tool (TrendCare®), but also recruiting additional
resources, and then expressing interest in being assessed as ready to have the
programme implemented. CCDM is not a mandatory programme, and the SSHW Unit is
reliant on the DHBs to progress the components of the programme, some of which are
time-consuming and resource intensive.
The timeframes for the roll-out of components of the programme are presented in Table
4. These are predicted by the SSHW Unit as the ideal. In reality, none of the DHBs have
completed a whole of system roll-out yet, but some are close.
Recommendations for re-ordering and creating efficiencies in the roll-out process are
discussed later in the document.
Final CCDM Evaluation Report (January 2015) 31
Table 4 Predicted phasing and timeframe for a DHB wide CCDM Roll-out.
CCDM Roll out in a DHB hospital Year 1 Year 2
Activities Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep
TrendCare implementation
Reliable TrendCare data
Negotiations & submission of EOI
Planning CCDM implementation
CCDM 3 hr Start up Work shop
Discovery process
All of staff survey
Interviews
Project/activity stocktake
T rendcare audit
Recruitment of site co-ordinator
Establish Organisational CCDM Council
Programme plan agreed and Letter of Agreement
signed
Mix and Match part 1 (for one ward)
Baseline measurement
Ward led action plan on findings
Local data council established
Make agreed changes
Mix and Match Part 2 (for one ward)
FTE calculation
Report with recommendations
Make changes based on recommendations
Variance Response Management
Churchill exercise
Capasity at a glance screen
Integrated operations centre
Variance indicator scoring
Standard operating response
Core data set established
Ability to resource according to base plan
Final CCDM Evaluation Report (January 2015) 32
Evaluation Findings
This section of the report systematically provides the final evaluation findings. Because
the programme was never static, with development and deployment continuing
throughout the 12 month evaluation process, the Programme Logic Model framework
that guided the evaluation process was also chosen as the most systematic way of
presenting the findings. Therefore, the focus of each section is framed under the
following headings:
1. Planned impact of CCDM
2. Resources supporting CCDM
3. CCDM interventions deployed, with a review of each component and process of
the programme
4. CCDM programme outputs, including the uptake of the programme
5. CCDM programme outcomes
6. Impact of the programme overall
7. Discussion of the evaluation findings
8. Recommendations for the future of the programme.
An Evaluation Findings Summary comments box is presented at the end of each section.
Planned Impact of the Programme
As outlined earlier in this report, the CCDM Programme emerged in response to the
findings of the SSHW Committee of Inquiry Report (2006), which ‘represented a shared
commitment by the New Zealand Nurses Organisation and the DHBs to work together to
agree on:
A mechanism for nurses, midwives and employers to respond immediately if
workloads exceed the determined levels
Sustainable solutions to safe staffing issues, developed in a way that has the
confidence of nurses and midwives.’ (2006, p7)
The suite of activities and tools developed and available within the CCDM allows nurses
on the front line, for the first time, to accurately challenge historical staffing and skill mix
levels and provide a more detailed and reasoned argument for a specific staffing
resource allocation throughout the 24 hour day. This programme also provides tools to
enable their work on the day to be carried out safely even in the case of unexpectedly
high workload. They have agreed strategies to manage an increase in demand. This
process has been developed by DHBs and unions working cooperatively to provide a
safer workplace for nurses, midwives and their patients.
The workload analysis and FTE calculation components of the programme are pivotal to
the review and redesign of staffing rosters that have, to this point, been based on
tradition or simple historical utilisation rates. This programme provides the opportunity to
use the evidence from validated tools to introduce models of care, skill mix and shift
lengths that more truly reflect patient need and enable a higher quality of hospital care.
Evaluation Findings
Based on feedback and reviews of other possible strategies to achieve these outcomes,
the evaluators concluded that the CCDM programme as a whole represented the first
step in a nationally consistent, fair and valid process to review and realign the nursing
and midwifery workforce in hospitals, so as to more truly reflect the levels of patient
acuity on the day. Furthermore, the programme provides a nationally standardised and
professionally agreed set of tools to monitor and respond safely and immediately to
unexpected changes in variance between workforce availability and patient demand
(acuity).
Final CCDM Evaluation Report (January 2015) 33
While the programme as a whole represents an ideal suite of tools and activities to
achieve the outcomes called for in the SSHW COI Report (2006), the patchy
implementation of the programme in DHBs to date has made evidence of this potential
more difficult to quantify. However, there were some very positive signs and evidence of
the impact being achieved in some more mature programme roll-outs that will be
presented in the outcomes section of the report.
Resources Supporting the Programme
The Safe Staffing Healthy Workplaces Unit
The Unit, established in 2006 with three years of funding ($400k per annum) by the DHBs
jointly, is staffed by a Director and five consultants who each take responsibility for
programme implementation in a number of DHBs. In 2011 a further two years of funding
at the same level was approved. During the evaluation period a further two years was
approved. A new Director was appointed at the beginning of the evaluation period,
which has seen a degree of consolidation of the programme take place up until the
present time.
As shown in Table 3 (above), the SSHW Unit work focuses mainly on training DHB staff to
apply the tools within their hospitals and wards. Over the five years of development and
implementation of the programme, the consultants, who have remained relatively
stable, have built up considerable knowledge about the programme and tools that
could be used more actively for the DHBs.
Evaluation Findings: The evaluators concluded that the SSHW Unit consultants, with their
thorough understanding of the programme and its tools, held the key to a more efficient
roll-out. They are better skilled to undertake some of the CCDM diagnostic activities for
the DHBs, rather than train DHB staff to undertake them, particularly the workload
analysis (Mix and Match Part 1) process and report, and the staffing FTE calculation (Mix
and Match Part 2), which are complex. Further, provision of this service centrally would
enable more efficient and objective analysis of these components of the programme,
and potentially national benchmarking data for DHBs to use.
DHB resources required to implement and roll out the programme
1. Installation, training and implementation of a validated, electronic patient acuity tool
such as TrendCare® software
A validated patient acuity programme is a vital prerequisite for CCDM. In the case of
TrendCare®, the only currently available tool which is used by all CCDM participating
DHBs, implementation includes:
additional IT resources to install TrendCare® and ensure the required feeds from
the hospital PMS and other sources are functional
recruitment of a TrendCare® co-ordinator
staff training workshops
additional hardware for nurses to use and view the programme within the wards.
2. Additional DHB staffing required to support the CCDM implementation
The CCDM programme requires resources, including the following DHB roles and
activities:
A CCDM Co-ordinator. While some use the TrendCare® Co-ordinator, who often
works part time in that role, a full time dedicated CCDM co-ordinator role is
recommended by the SSHW Unit, particularly during the implementation phases.
Final CCDM Evaluation Report (January 2015) 34
Business analyst time to support the work on the ward Core Data Set and on the
workload analysis and FTE calculation (Mix and Match Parts 1 and 2), and the
resulting budget implications. This is vital, but is often not overtly factored in.
IT support to ensure that TrendCare® remains functional, and to support the
development of the hospital at a glance screen(s) and delivery of automated
data sets and reports.
The DON, Charge Nurses/Nurse Managers and staff in the wards undergoing the
Mix and Match activities need to factor in downtime for these activities.
The roster re-engineering, including skill mix and shift changes, needs to involve
negotiation with HR and unions, as well as with the executive team, if FTEs need to
be recruited or reduced.
3. Computer hardware and electronic display screens
Providing staff with sufficient computers to access TrendCare® and display screens to
provide the ‘hospital at a glance’ increased the transparency of the workplace, as well
as providing staff with opportunities to become more numerically literate.
4. Executive team and nursing leadership time commitment to CCDM
The intent of the programme was that there be on-going commitment and involvement
of the executive team in the CCDM implementation process. There was a particular
expectation that the DON and DOM become champions of the programme. This
required dedicated time over the implementation and roll-out period.
5. Additional staffing resource at ward level following FTE diagnostics
The key staffing diagnostic tools in the CCDM programme, the workload analysis (Mix
and Match Part 1) and FTE calculation (Mix and Match Part 2) both have the potential to
identify under-resourcing of staff at ward level and the need to increase the FTE. This has
been put forward as a reason for there having been a slower uptake of the FTE
calculation in particular.
6. Roll-out of Productive Ward: Releasing Time to Care (RTC).
While this programme is not a necessary prerequisite to CCDM, many of the New
Zealand DHBs did take up the programme, encouraged by the Ministry of Health, which
had purchased licences. It was the observation of the evaluators, and based on
feedback from DHBs and SSHW Unit consultants, that rolling out CCDM to a DHB that had
already implemented the productive ward series made the process easier. The three
major components of the RTC programme were built on by the CCDM programme:
Knowing how we are doing (ward performance benchmarking) is incorporated
within the ward core data set.
Well organised ward (simplification and streamlining the workplace) enables a
more valuable workforce analysis (Mix and Match Part 1) because the ward
clutter will have been eliminated.
Patient status at a glance (provision of summary patient information on a board
for quick reference) is built on to incorporate the hospital at a glance with
information on the status of each ward, including occupied and spare beds as
well as acuity status.
Being able to piggyback on these initiatives was seen as having the potential to save
additional resources.
Evaluation Findings
All DHBs had underestimated the resources required to implement the full breadth of the
programme. In many instances, especially in the early implementation phase, the
resource required for DHBs rolling this out to all wards and departments in the hospital did
not become evident until they were further into the process.
Final CCDM Evaluation Report (January 2015) 35
Impact of under resourcing the programme’s implementation
In line with the findings of Moore and Blick’s (2013) assessment of a whole of hospital
programme roll-out of the Productive Ward programme, the CCDM evaluators found
that slow uptake and reduced resources at DHB level were mutually dependent. If there
was a slow down at any point in the roll-out, a shift in resources away from CCDM was
often undertaken, which then led to a further slowdown and more difficulty reenergising
the programme further down the track. The sequencing of activities should occur so that
dependencies are managed well and the implementation flows.
Impact of refocusing resources
Many of the DHBs chose to second their TrendCare® Co-ordinators to the role of CCDM
Co-ordinator, leaving the previous role unfilled. Because of the time and focus required
by the CCDM Co-ordinator, the oversight and in-house support for TrendCare® was no
longer available. When time came for CCDM implementation at ward level, it was
found, in many cases, that the TrendCare® actualisation rates and interrater reliability
rates were not even close to the required 100%. Each ward then required a
considerable amount of resource to improve the quality of their TrendCare® data, which
was an unexpected obstacle when initially looking at the implementation of CCDM. In
some DHBs implementing CCDM has required a re-launch of TrendCare®, causing roll-
out slippage, with the data integrity compromised.
Another issue related to some DHBs believing that CCDM had become ‘business as
usual’ too early, and they therefore redeployed or downsized the CCDM Co-ordinator
role. It may be necessary to keep a CCDM co-ordination role in place even when the
programme is ’business as usual’, as was found in the case of the TrendCare® Co-
ordinator role. Slippage was noted in the Assessment of the Productive Ward
Implementation (Moore and Blick, 2013) when the dedicated resource for
implementation ceased.
Impact of workload analysis and FTE calculation on staffing resources in the wards
There seemed to be a reluctance to undertake the Part 2 FTE analysis, which centred
around the potential for understaffing to be identified, and then to need to be
addressed, which would be costly. Conversely, the thought of needing to reduce staff or
alter the skill mix and rosters was also daunting.
The evaluators examined the Part 2 reports from three DHB showcase wards. In two
wards they evidently had been flexing up existing staff in an ad hoc manner, and were
over budget significantly. Following the FTE analysis, the exact amount of additional staff
required was identified; this resulted in a more stable roster, with the additional FTE equal
to the actual FTE being utilized regardless. Through this process, overtime and use of
casuals reduced the degree of over expenditure. In a third example, the ward
addressed skill mix and FTE reconfiguration to implement additional cost neutral FTE.
One DHB chose to use a ‘no surprises’ approach when applying the FTE calculation, by
including all wards collectively, which is possible with TrendCare® data, in order to
identify the ‘whole of hospital’ FTE deficit or surplus. Data from hospital-wide Part 2
calculation would inform the executive team as to whether they needed to recruit to the
hospital or were over capacity. This particular DHB found that their overall FTE seemed to
be appropriate, but that there were unders and overs in various wards. This would result
in a redistribution of staff over the whole hospital, while maintaining the same overall FTE.
CCDM Interventions Deployed
The vision set by the SSHW Committee of Inquiry (2006) was for a whole of organisation
approach to achieving safe staffing and a healthy workplace for nurses and midwives,
with the ultimate goal of providing safe, effective and cost effective care for patients in
a timely manner. They warned that it would be a lengthy process (estimating at least
three years).
Final CCDM Evaluation Report (January 2015) 36
Organisational engagement processes
The SSHW Unit describes the conditions for programme implementation as follows:
DHB Executives agree that this is a priority.
A validated acuity tool has been used in the hospital for the previous 6 – 12
months (TrendCare® is the only one currently used).
There is willingness of the DHB to work in partnership with the local unions and
SSHW Unit throughout the process.
There is willingness to recruit and resource an on-site co-ordinator to implement
the programme.
There is partnership and agreement between unions and DHBs.
It is not until approval is received from the SSHW Unit Governance Group (after the
Discovery process) for the DHB to implement the programme that ownership of the
process broadens to include the identification of suitable DHB employed NZNO
delegates to partner in the programme roll-out and the development of the
representative DHB CCDM Council.
While initial interest in the programme was usually generated at CEO, DON, COO and
GM level, the readiness of wards to engage in the process at the initial stages of
implementation was less evident. The evaluators would describe CCDM as ideally a
‘middle out’ approach. Whilst there is necessary ownership of the programme at
executive level, and also union buy in, the greatest impact of the programme in terms of
change management and the application of complex tools and processes occurs
mainly at ward level. Success is then dependent on the leadership skills, experience and
‘mana’ of the Charge Nurse Manager (CNM). The CCDM activities require management
leadership and change management skills. There is potential for slow down and bottle
necks relating to a general resistance to change. The preparation of the CNM for such a
pivotal role in the programme roll-out at ward level had not previously been evident.
Once the programme has been approved for implementation, a series of processes and
tools are used to gauge the organisation’s readiness for the programme in the Discovery
phase. This is undertaken by the SSHW Unit consultant, and includes:
Key staff interviews
On–line SSHW staff survey
The TrendCare® Audit Report
Stocktake of current DHB projects and programmes
On completion, a Discovery Report is written and tabled with the CCDM council. The
report contains a summary of the staff interviews and surveys, an action plan in response
to this, a stocktake of current DHB projects and programmes focusing on improving
capacity and demand management, flow, information and service quality, and an
agreed site plan for the next 12-24 months.
Evaluation Findings:
This Discovery phase is planned to take place over a 6–8 week period, but feedback to
the evaluators indicated that the process is much more time consuming if all of the
information required is to be obtained. Also there was concern that the surveys often
clashed with other DHB-generated workforce surveys, hence there was generally a poor
response.
The reports also took some time to compile and were, given the breadth of information
collected, very complex and time consuming to work through.
Concern was also expressed that these processes raised expectations that CCDM
related activities at ward level were imminent. However, there was a likely delay of some
months before the reports and action plan were presented to the CCDM Council, and
even then the DHB might not be ready for implementation.
Final CCDM Evaluation Report (January 2015) 37
There was generally a long time lag between the Discovery phase and CCDM
implementation, which could potentially derail further progress. Benefits could be
gained from having a more efficient process to prepare and feed Discovery reports
back to the DHBs.
Programme implementation
Establishing programme governance
Once the decision is made to implement the programme, a Letter of Agreement is
signed between the DHB, NZNO, PSA and the SSHW Unit Governance group. Then the
CCDM Council is established, the CCDM co-ordinator is recruited and an education
programme is undertaken by the SSHW Unit for those initially involved in the
implementation, including the DHB business analyst. Figure 5 below outlines the
organisational overview of the CCDM programme within the DHB.
Figure 5. CCDM Governance and organisational overview at DHB level
Evaluation findings:
This governance and operational structure to support the programme within the DHB is
broad and complex. It was not clear whether any of these groups or roles could have
been integrated with others within the DHB. It seemed that the roles and activities were
additional to all but the CCDM Co-ordinator, who was resourced for the role.
Evaluators found that during the initial stages of the implementation, and when the SSHW
Unit consultants were frequently active in the early stages of implementation, the
councils met regularly and were well attended. However, as the programme roll-out
inched along ward by ward, the structures began to loosen and required skilful
leadership, generally by the DON, and the enthusiasm of the CCDM Co-ordinator, to
keep the momentum up. This is evident by the number of active ‘local data councils’ in
the DHBs at the time of this final report, only 17 throughout the 11 active DHBs.
The evaluators noted that although the CCDM Co-ordinators were mainly employed in
that role part time, generally they worked well over the hours allocated in order to
maintain the programme’s momentum. When asked about the level of support available
through the SSHW Unit consultants, the evaluators were told by the SSHW Unit that the
consultants were in the DHBs by invitation, and did not have any mandate to actively
progress the programme roll-out in a DHB.
Central DHB CCDM (partnership) Council
CCDM Co-ordinator
DHBs in partnership with unions
SSHW Unit
SSHW Unit Consultant Central
operations management Participating wards
Ownership
Governance
Secretariat and operations
Mental Health Data Council
ED Data Council
Paeds Data Council
Maternity Data Council
Medical Data Council
Surgical Data Council
Final CCDM Evaluation Report (January 2015) 38
Establishing the Core Data Set
A critical role for the DHB CCDM Council was to support the development of the Core
Data Set (CDS), described as an agreed set of indicators from the clinical floor to the
Board. Some of the required metrics are already collected by the organisation (such as
harm markers, and indicators related to resourcing), with the plan being that others will
be progressively introduced as they are developed and tested. The SSHW consultant
supports the Central CCDM Council through a process of review of the recommended
indicators, and a determination of their current status within the organisation. The Central
CCDM council and local councils then determine which of the indicators can be
produced from existing data systems, and how this might be done to reduce the burden
of data collection. They also determine a starting place for the scrutiny of data by the
CCDM Council and develop an organisational plan to have all indicators collected,
accurate and available in a central repository, with an appropriate reporting format that
all can access and engage with. Facilitating the process outlined above and coming to
an agreement on the metrics take significant time and were often not achieved.
Evaluation Findings
At the time of the final evaluation report, only four of the 11 DHBs had agreed on a Core
Data Set. This meant that other DHBs had invested in the programme without agreed
measures of its effectiveness and impact. It also made it difficult for the evaluators to
easily obtain quantitative evidence of the programme impact for this evaluation.
The Health Quality and Safety Commission initiative focusing on the development of NZ
wide Health Quality and Safety Indicators (http://www.hqsc.govt.nz/our-
programmes/health-quality-evaluation/projects/health-quality-and-safety-indicators/)
and current DHB reporting requirements should provide clear guidance on the
development of a nationally agreed set of core indicators against which the impact of
the programme can be measured.
Developing a system platform
Use of a validated electronic patient acuity and workload management software
solution for at least six months prior to CCDM implementation is a prerequisite for the
implementation of the programme into a DHB. While the programme is vendor agnostic,
currently TrendCare® is the only validated tool available. Therefore, all participating
DHBs were using TrendCare® which was first introduced to New Zealand in 1999. By 2013
it was being used by 15 of the 20 DHBs, covering 5,550 inpatient beds in 34 facilities.
Table 6. The components of TrendCare® used by the DHBs as at September 2014
Dis
tric
t
He
alth
Bo
ard
s
Tre
nd
Ca
re®
ve
rsio
n
Ro
ste
r sy
ste
m
Sta
ff a
lloc
atio
n
Pa
tie
nt
ac
uity
Ac
uity
ind
ica
tors
Ass
ess
me
nts
Ca
re
pa
thw
ays
Hu
ma
n
Re
sou
rce
ma
na
ge
me
nt
Dis
ch
arg
e
an
aly
sis
Tim
e o
ff in
Lie
u
Pa
tie
nt
ha
nd
ov
er
Die
t o
rde
rin
g
Drs
list
Mu
lti-C
am
pu
s
Alli
ed
He
alth
Northland 3.4
Waitemata 3.4
Bay of Plenty 3.4
Tairawhiti 3.4
Taranaki 3.4
Whanganui 3.4
Hawkes Bay 3.4
MidCentral 3.4
Wairarapa 3.4
Hutt Valley 3.4
Nelson/Mar 3.4
West Coast 3.4
Sth Canty 3.4
Southern 3.4
Final CCDM Evaluation Report (January 2015) 39
Ideally, the DHB needs 12 months (6 months minimum) of reliable data from TrendCare®
prior to implementing CCDM. This includes achievement of patient hours per patient day
(HPPD) within benchmark, inter-rater reliability of over 90%, actualisation compliance of
100% (all clients are in the system and information is regularly updated) and ensuring that
the Allocate Staff screen is accurate, having been audited as such twice a month for
two months.
The information from the TrendCare® system is used to inform many of the CCDM
processes, including:
mapping the organisation
feeding into the Core Data Set for monitoring and reporting
workload analysis benchmarking
FTE calculation to determine the base staffing
populating the ‘hospital at a glance’ screen
providing information to inform variance response management
populating the data required by the ward for on the day management and
monitoring
populating the screens in the integrated operations centre to enable a whole of
hospital (and DHB if all hospitals are using TrendCare®), with a regularly updated
feed providing a view of patient flow (emergency, theatre and bed occupancy)
as well as the associated staffing in these areas
providing a suite of reports that enables tracking, monitoring and benchmarking of
CCDM activities within the DHB (and potentially between DHBs).
To date the product has a number of features that make it more functional for the
CCDM Programme roll-out including:
currently it is used in the majority of DHBs in the country (16/20)
is currently the only validated electronic patient acuity tool, a CCDM requirement,
has nursing HPPDs benchmarked against over 150 patient types nationally and
internationally to achieve consistency, enabling DHB benchmarking,
is well supported by the company with a well-developed standardised training
programme for DHBs and a responsive software support service which is mature.
The evaluators were aware of one US product that is in the early stages of development
in NZ; however, the acuity is not validated. Currently, the product is also being adapted
(hybridised) by site, unlike TrendCare®, where there is a nationally standardised product
and all DHBs are on the same version.
Evaluation Findings
The evaluators did question the reliance on one software system for such significant
components of the CCDM programme. The evaluators met with this software
developer and another vendor claiming to have a similar product. They also received
feedback from users on site visits throughout the country and from the CCDM
consultants to better gauge the value of the TrendCare® product to the programme.
Risks associated with reliance on this one product include:
- it is not web based, so upgrades need to be managed DHB by DHB, with no pressure
to ensure upgrades are made available
- the SSHW Unit is co-ordinating the development of additional features for the
programme, including maternity and community care, but there is no pressure for
the vendor to make the changes and provide associated upgrades
- any mandating of the CCDM programme for all DHBs would place the vendor in a
position where they would have a monopoly and could significantly increase the
cost of the product
- currently each DHB has its own licensing arrangements with the vendor.
Final CCDM Evaluation Report (January 2015) 40
Getting the base right
The methodology used for this component of the process consists of the two Mix and
Match tools: Part 1 workload analysis, and Part 2 FTE calculations. The aim is to better
match workforce resources with patient acuity and demand patterns, as well as ensuring
that the qualifications, knowledge and experience of those providing the service are
fully utilised. In other words, the team providing care on the day needs to be matched
with the specific care needs of patients on the day.
There are certainly challenges in managing an acute hospital where the day stay has
been reduced over time, resulting in only the more complex, dependent patients
requiring hospital care. Maintaining a balance between resources available and those
required has vexed health management over more recent times. However, there is little
attention paid to the impact of the workload associated with patient churn (admission
and discharge) and the amount of time it takes for nurses and midwives to attend
rounds, meetings, develop protocols, maintain care (including medications and personal
cares on time) and prepare patients for discharge or orientate the new patient and
family.
The CCDM philosophy would state that unless you have the base staffing right to match
the patient activity and acuity need on the day, there will always be inconsistencies in
patient flow, and quality of care will be in jeopardy. Once the appropriate/safe base
staffing has been calculated and variance is being predicted and consistently matched,
interventions to facilitate or change ‘patient flow’ will be more responsively managed,
and adaptation to the roster and team skill mix will be made as and when appropriate.
Workload analysis (Mix and Match Part 1)
In its current form, the workload analysis process is similar to a time and motion study. A
standardised workload analysis tool is used to cover a two week period when each staff
member records, every 15 minutes, the activities they have undertaken in that time. This
is analysed to identify the work performed in that period, what was unable to be
satisfactorily completed (care rationing), the degree to which staff undertake non-core
activity (not directly related to an individual patient), and what the activities are made
up of. It also provides an opportunity to collect evidence on work patterns and activity
peaks and troughs, and activity currently being undertaken but not accounted for in
NHPPD, as well as areas of opportunities for improvement to skill mix/base staffing/
rostering/ processes.
Analysis of this process then helps the ward to understand the required skill mix, clinical
and non-clinical activity and appropriate schedule to match predicted patient demand
and staff activity patterns, referred to as roster re-engineering.
The information provided by this report may assist the DHB to identify some areas of
opportunity for improvement. For example, these may include the scheduling pattern of
the ward, optimisation of the model of care, review of the way in which work is currently
organised and review of identified organisational policies and practices to support a
reduction in workload, leading to greater overall efficiency gains. The table below
indicates that the ward by ward roll-out of the workload analysis has been slow.
Final CCDM Evaluation Report (January 2015) 41
Table 7. Number of wards with workload analysis applied and changes made
DHBs September 2014
Year
commenced
CCDM
Wards for CCDM
implementation
Workload
analysis
Changes
based
workload
analysis*
Bay of Plenty (RTC) 2009 9 9 3
Northland 2010 14 10 2
MidCentral (RTC) 2010 10 6 4
Nelson/Marl 2010 6 5 1
Tairawhiti (RTC) 2011 4 4 4
Taranaki (RTC) 2011 6 1
Southern 2011 20 11 3
Waitemata (RTC) 2012 10 3 1
Hutt (RTC) 2013 8 1
Sth Canty (RTC) 2013 4
Whanganui (RTC) 2013 4 2 1
Total wards 85 51 19
Wards completed 60% 22.4%
*Some changes may not need to be made following the workload analysis process.
Evaluation Findings
The workload analysis is time consuming and work intensive for staff. It requires significant
DHB resources for data entry and analytical time to work through the staff activities
forms. The final report, also compiled by the site co-ordinator, then takes some time to
complete, because of the logistics required in translating the findings from the workload
analysis to a report. It is estimated that the time frame from workload analysis to report is
about six months, by which time the enthusiasm generated out of the data collection
process has dissipated. Application of the recommendations takes significantly longer.
The strongest criticism at ward level centred on the length of time it took to produce the
report. Staff were eager to view the findings, and it was suggested that a quick
turnaround of the report would have enabled changes to have been implemented with
more enthusiasm.
By September 2014, only 51 of the 85 potential wards throughout the active DHBs had
completed workload analysis, and only 19 (22.4%) had implemented recommended
changes.
Suggestions were made to the evaluators, and are being followed up by the SSHW Unit
to develop a software programme for the workforce analysis study, and for the Unit to
take a more active role in managing the process efficiently, expediting the production
of the report and negotiation over changes.
The evaluators had queried the necessity of the workload analysis8, but were convinced,
during the site visits with staff who had participated, that this was one of the most
profound activities they had gone through. They felt that management at last was being
provided with evidence of the complexity of the average day for a clinician and the
variety of activities they undertook. In one hospital the medical and allied health staff
insisted that they also take part in the process, which was a key lever in winning staff
over. The SSHW Unit consultants also supported the staff view that this process won them
over to CCDM as a programme.
To identify the true nature of the Mix and Match Part 1 workload analysis, it was
recommended by many, including the SSHW Unit Consultants and Director, that the title
‘workload analysis’ is more appropriate, hence this process has been referred to as such
in this evaluation.
8 This activity involved staff recording all of the activities they had undertaken during set time
periods.
Final CCDM Evaluation Report (January 2015) 42
Full Time Equivalent staffing (FTE) calculation (Mix and Match Part 2)
The second part of the Mix and Match process is the calculation of FTE needed in the
ward throughout the day and over time to match patient demand patterns and
minimise variance. It involves analysing the HPPD data for each day and each shift over
a 12 month period. This includes both the direct and indirect clinical hours required to
give a total required HPPD figure. This then enables calculation of the FTE requirement,
including clinical and non-clinical FTE, to ensure optimum skill mix and schedule,
supported by the Part 1 analysis, to match the predicted patient demand and staff
activity. MECA leave averages and entitlements for education and annual leave are
also included in the formula.
Other resources used to validate this calculation include:
The work analysis report for the ward, if one has been carried out
TrendCare® reports, each for the same 12 month period
Relevant collective agreements for staff working in the ward/unit, e.g. District
Health Boards / NZNO Nursing and Midwifery Multi-Employer Collective
Agreement (MECA)
Organisational protocol regarding professional development, orientation and
supernumerary time, and Nurse Entry to Practice, if the organisation has these
Information on additional competencies or recertification requirements for which
regular training is required for staff on the ward
Organisational averages or other data on leave taken, e.g. sick, bereavement,
maternity, parental, long service
Whether any leave types within the organisation are not included in the
ward/unit’s budget, e.g. maternity or parental
12 month turnover statistics for the ward/unit, or if this is unavailable, the average
organisational turnover for the previous 12 months
Number of new graduates employed by the ward/unit per year
Current budgeted rostering pattern for the ward/unit.
Currently the DHB staff need to complete this analysis with a high level of training and
support from the SSHW Unit staff, who provide the following standardised set of resources;
a Workbook Template (Excel file), a New Roster Profile Worksheet (document), Available
Hours of Staff and Variance Response Tables (documents) and a Mix & Match Workbook
(reference workbook).
The information from both parts of the Mix and Match process is necessary in order to
calculate a realistic staffing budget for the service.
Again as with the workload analysis process, the time and complexity involved in the FTE
calculations has resulted in very slow progress with this tool. See the table below.
Final CCDM Evaluation Report (January 2015) 43
Table 8. Wards with FTE calculation applied and changes made
DHBs September 2014
Year
commenced
CCDM
Wards for
CCDM
implementation
FTE
Calculations
Changes based
on FTE
calculations
Bay of Plenty (RTC) 2009 9 9 9
Northland 2010 14 1 1
MidCentral (RTC) 2010 10 4 2
Nelson/Marl 2010 6 3 3
Tairawhiti (RTC) 2011 4 4
Taranaki (RTC) 2011 6 1
Southern 2011 20 6 3
Waitemata (RTC) 2012 10 2
Hutt (RTC) 2013 8 3
Sth Canty (RTC) 2013 4
Whanganui (RTC) 2013 4
Total wards 85 33 18
Wards completed 38.8% 21.2%
*Some changes may not need to be made following the Workload analysis process.
Evaluation Findings
Because FTE calculation was identified as Mix and Match Part 2, it was generally
sequenced following the Mix and Match Part 1 workload analysis. However, discussion
with the SSHW Unit consultants and Director, supported by feedback from some DONs,
indicated that inclusion of all wards in a hospital-wide FTE calculation, irrespective of
whether the workload analysis has been completed, provides a better picture of a
hospital wide FTE deficit or surplus and allows targeting and prioritising of the workload
analysis. In fact it could be argued that the FTE calculation could be used to prioritise
which wards need to undergo a workload analysis with urgency. The FTE calculation is
also a useful tool to support negotiation of resources to enable hospital-wide roll out of
the CCDM programme.
The complexity of both the FTE calculation and the subsequent report was identified as
another reason that few wards (38.8% of all wards) had completed this process. The
evaluators agreed with DONs and the SSHW Unit staff that expediting the FTE calculation
and having this analysed centrally by the SSHW Unit would expedite the roll-out of the
CCDM Programme, particularly if the process was used to prioritise wards for workload
analysis.
Implementation of Variance Response Management
The Variance Response Management (VRM) system is designed through the CCDM
programme development as a set of elements that fit together to create a
comprehensive response safety net. The CCDM VRM system includes a suite of seven
tools designed to assist the DHB to manage variance to maximum effect. These tools
were designed in conjunction with Bay of Plenty DHB, and function as a whole of hospital
process. They are:
1. The Churchill exercise
This exercise enables all staff to obtain a big picture view of an actual day in the
organisation and introduces the language of the CCDM programme, specifically of
VRM, as well as assisting staff to identify their role in alerting, responding to and reporting
variance.
The Churchill exercise is facilitated by the SSHW Unit using a standardised format. The
DHB’s current VRM strategies are presented and potential areas for development
identified and discussed. This shared understanding can then form the basis of any
activity designed to improve variance management as part of the implementation of
the CCDM programme.
Final CCDM Evaluation Report (January 2015) 44
The Churchill exercise, taking 4-6 hours to run, requires as many representatives as
possible from the following areas; clinical staff (e.g. nursing, allied, medicine, radiology),
operational staff (duty managers, service managers) and executive staff (CEO, COO,
DON, CFO, CMO). The SSHW Unit provides a facilitator (usually a programme consultant),
props for the table top exercise (patients, staff, traffic light flags, VIS lists), data about the
day that is collected in advance by the CCDM Coordinator, a PowerPoint overview of
the VRM system, and a summary of what has been learned by other DHBs implementing
the VRM system.
The DHB provides staff released to attend the exercise, a conference room sufficient to
accommodate tables and between 40 - 70 people, enlarged floor plans of each
ward/unit, including ED, theatres, OPD, and recording of the session or interviews
conducted before the day.
Evaluation Findings
The Churchill exercise has been described as a ‘light bulb’ moment for many clinicians in
their understanding of VRM from a hospital wide perspective. It is a valuable part of the
set of tools in this regard, but it does have limitations. A significant resource is required
from each DHB for the exercise, in terms of the numbers of managers and clinicians
needing to attend to gain hospital wide benefits from the exercise. If the format for the
exercise was adapted and, for example, the Churchill exercise was videotaped and
uploaded, this format would allow for all newly recruited staff to gain awareness as well.
2. Capacity at a Glance (CaaG)
Capacity at a Glance (CaaG) is a patient management system and TrendCare®-driven
view of all units of the hospital (or the DHB). The CaaG (also known as Hospital at a
Glance, HaaG) view illustrates physical capacity, patients by type and number, and the
match between demand and capacity. This is identified as an optional tool, which the
DHB needs to resource and manage its development. By September 2014, nine of the 11
DHBs had developed the tool and eight had it displayed hospital wide on large screens.
The screens represent data in real time. They are used both at ward level by nurses to
obtain an overview of hospital ward status, and at hospital level for other departments,
including emergency, duty managers and executive management to inform the
decision making required for smooth patient flow. Figure 6 below illustrates the screen
shot. The screen designs are currently custom made by each of the DHBs.
Figure 6. Sample of a Capacity at a Glance screen
Final CCDM Evaluation Report (January 2015) 45
Features displayed on the screen build on those developed by the productive ward:
releasing time to care programme. The screen shot above indicates a ward’s
occupancy by service and patient outliers from their average length of stay. Some
HaaG screens indicate by ward the variance in HPPD required versus actual.
Evaluation Findings
The data transparency of the CaaG screens led to a culture change in many wards,
whereby nurses started to view themselves as a part of a wider hospital rather than a
siloed ward. They were able to see when other wards were busy and offer support and
vice versa. The culture towards deployment improved with the transparency of the data,
which in turn led to a more generally skilled workforce.
3. Variance Indicator Boards
This tool, originally titled ‘clinical status in real time’, enables staff on the floor to indicate
their clinical status (impact of the match between demand and capacity) in real time.
The DHB CCDM Council agrees to a set of indicators in principle, and each service then
personalises its own Variance Indicator Board (VIB). Real time clinical status will appear
on the Capacity at a Glance view/screen. Business rules and end user training are a key
requirement to the success of this tool. These are updated at the beginning and end of
each shift, as well as any time there is a change in clinical status. This is generally by the
shift co-ordinator. While it may seem that a significant amount of subjective judgement is
made in responding to the cues in the VIB, the judgement of professional clinicians in the
setting is an important element of decision making. Additional contextual data
augments the reliability and validity of this tool on the day.
Figure 7. Variance Indicator Board
Evaluation Findings
The ability for staff on the floor to be able to quantify their workload through this
mechanism at the start and end of a shift has had mixed feedback. While it has been
helpful in regards to increasing transparency of work, sharing of information and data
capture between wards, feedback has also signalled the lack of willingness of some staff
to enter in this data and therefore complete the process. However, it can be argued
that the process is relatively simple and can be reasonably quick to complete. Focus
should be given to helping staff better understand the rationale for this, and how the
information ultimately assists them with being able to better understand and quantify
their workloads and subsequently gauging overall ward acuity.
Final CCDM Evaluation Report (January 2015) 46
4. Standard Operating Responses
The Standard Operating Responses (originally titled Desk Top Traffic lights) plans contain
the response to variance strategies available to each unit or level of the organisation.
The traffic light status is determined by the score from the VIB. These are designed to
develop standardised responses to variance within units and across operational and
service levels of the organisation. This allows for an assessment of how effective response
strategies are on the day and over time. Development of the response system is based
on this data. The Capacity at a Glance view/screen will be updated with the
corresponding colour when changes are made to the Variance Indicator Board at ward
level.
Figure 8. Standard Operating Responses framework
Evaluation Findings
Essentially the frameworks are similar to many DHBs’ escalation plans, and initially there
was some question as to whether this requirement was a necessary part of the CCDM
programme. The benefit of maintaining the Standard Operating Responses within the
CCDM programme is two-fold. Firstly it dovetails with the VIB and redeployment policies.
Secondly it allows for a national standardised approach (with local variation) to a ward
escalation plan.
Reporting can identify what responses have been followed accordingly when a ward
reaches orange and red status.
The wards reported entering gridlock or red less often after introducing the responses
framework, as action was taken at a lower orange level to prevent going into red.
5. The Reallocation (redeployment) Policy
The purpose of the Reallocation Policy is to apply a consistent process to the experience
of being moved between/within services to meet increased demand in another part of
the hospital (this should not result in an increase in reallocation activity). There are an
agreed set of tasks that a nurse who is on short term (two hours or less) could undertake
safely without needing a full handover and patient allocation.
The CCDM Coordinator facilitates the negotiation process to arrive at the agreed tasks
and coordinates trials and refinements to the tools. Information collected from the Smart
5 audits is used to identify improvements to the response on the day and over time.
Final CCDM Evaluation Report (January 2015) 47
Figure 9. The reallocation policy agreed short term redeployment tasks
(sample from one DHB)
Evaluation Findings
The smart 5 cards were seen in practice to be of assistance to DHBs introducing VRM and
deployment. Nurses who had been reluctant to deploy to wards outside their service (as
they feared being handed a patient load) were relieved that the cards acted to protect
them from inconsistencies with some charge nurses, who wanted to hand them a patient
load. Once nurses gained confidence practising outside their home ward, it was
reported that the general skill set increased and the DHB was able to deploy with much
less resistance. Where VRM had been implemented for a longer period of time and the
process had become ‘business as usual’, the cards were no longer required.
6. Essential Care Guidelines
The purpose of this framework is to apply an agreed decision-making process to those
occasions when sacrificing decisions need to be made regarding care activities (care
rationing), due to a surge in demand or deficit in capacity. A policy is developed by the
CCDM council (or appointee) to accompany the framework that describes the
conditions under which it is invoked and what reporting is required.
Data is collected on how often the framework is invoked and the outcome for patients,
staff and the organisation. This provides evidence for service redesign.
Final CCDM Evaluation Report (January 2015) 48
Figure 10. Essential care guidelines
Evaluation Findings
An agreed policy was found to act as a quality assurance tool, as it ensured that
essential tasks, such as dispensing medications and taking observations, were
completed.
7. Integrated Operations Centre (IOC)
The Integrated Operations Centre (IOC) provides a framework for active management
of variance on the day, as well as forecasting and planning in response to variance over
time. The requirements are a physical space with a Capacity at a Glance screen, multi-
disciplinary engagement at daily meetings, high value reporting and accountability for
planning and forecasting.
Attendance at daily ops meetings by the evaluators enabled a better understanding of
the potential for a central location where the hospital(s) as a whole could be monitored.
Bed managers and after hours duty managers reported that their role was significantly
enhanced through being able to locate themselves in this venue and receive a regularly
updated picture of the hospital as a whole, including patient flow.
Final CCDM Evaluation Report (January 2015) 49
Figure 11. A daily Operations Centre meeting including 2 outlying hospitals
Evaluation Findings
There are varying levels of maturity of DHB IOCs. Some are more advanced than others
and can offer a truly centralised coordination mechanism, as well as a venue for daily
operations meetings. In places where there has been obvious investment in the IOC as a
central hub for the hospital, there is a greater sense of a cohesive approach to system
integration. Ultimately each DHB should aim to have fully functioning IOCs, matched to
the size and requirement of their DHB. It is the overarching system integration and ability
to take the bird’s eye view of the DHB that has often been lacking in the way hospitals
have been traditionally managed. Adequate IOCs can assist with improvements in this
ability to run and manage hospitals as whole organisations, not merely as siloed wards,
independent of one another.
Variance response tools as a collective strategy
Overall, the Variance Response Management suite of tools was designed to provide a
collective strategy to safely and efficiently manage unexpected variance. As noted in
the table below, all DHBs which have implemented the programme for more than a year
have all three key components: the VRM strategy, the CaaG screen and the Variance
Indicator Board. Only four have the Integrated Operations Centre.
Table 9. Proportion of DHBs with CCDM variance response tools established
DHBs September 2014
Year
commenced
CCDM
Variance
response
strategy
Capacity
at a
glance
screen
Variance
indicator
board
Integrated
ops
centre
Bay of Plenty (RTC) 2009 Y Y Y Y
Northland 2010 Y Y Y N
MidCentral (RTC) 2010 Y Y Y N
Nelson/Marl 2010 Y Y Y Y
Tairawhiti (RTC) 2011 Y Y Y N
Taranaki (RTC) 2011 Y Y Y N
Southern 2011 Y Y Y Y
Waitemata (RTC) 2012 Y Y Y Y
Hutt (RTC) 2013 N N N N
Sth Canty (RTC) 2013 N N N N
Whanganui (RTC) 2013 N N N N
Total 8 9 8 5
% of DHBs/wards completed 66.7 75.0 66.7 41.7
Final CCDM Evaluation Report (January 2015) 50
Evaluation Findings
VRM as a collective strategy has been implemented in different ways throughout the
DHBs. Many DHBs reported that the barriers to VRM were often financial, due to the high
cost of CaaG/HaaG screens and the associated IT infrastructure requirements. The
components that were often well managed were the patient status at a glance screens
and variance indicator scorings, once resourcing could be secured and IT had the
capacity to deliver on the requirements. Standard Operating Responses were less
standardised, and varied in terms of actual implementation.
CCDM Programme outputs
This part of the evaluation focuses on the level of uptake of the programme at DHB level
and the spread of the programme to other DHBs.
At the time the evaluation commenced, 11 of the 20 DHBs were implementing the
CCDM Programme and another four were expected to begin implementation over the
following 12 months. During the evaluation, one large DHB commenced implementing
the programme.
Table 10 below identifies the 12 DHBs currently in varying phases of implementing the
programme. West Coast and Counties Manukau DHBs were two of the three pilot DHBs
initially involved in the programme development. Counties Manukau exited during the
pilot phase because they did not have an electronic validated patient acuity tool. West
Coast exited some time later, but still uses TrendCare®.
Table 10. Progress with CCDM roll-out from 2009 - 2014
DHBs
September
2014
Ye
ar
co
mm
en
ce
d
Cu
rre
ntly a
ctiv
e
Dis
co
ve
ry r
ep
ort
Loc
al D
ata
Co
un
cils
Mix
& m
atc
h P
art
1
Ch
an
ge
s to
w
ard
fo
llow
ing
M&
M p
art
1
Mix
& m
atc
h P
art
2
Ch
an
ge
s to
w
ard
b
ase
d
on
M&
M p
art
2
Co
re d
ata
se
t d
efin
ed
Va
ria
nc
e r
esp
on
se s
tra
teg
y
Ca
pa
city a
t a
gla
nc
e s
cre
en
Va
ria
nc
e in
dic
ato
r b
oa
rd
Inte
gra
ted
op
s c
en
tre
Bay of Plenty
*(RTC) 2009 y Y 3 9 3 9 9 Y Y Y Y
Counties 2009 N Y Y Y
West Coast
(RTC) 2009 N Y
Northland 2010 Y Y 1 10 2 1 1 Y Y Y Y
MidCentral
(RTC) 2010 Y Y 6 4 4 2 Y Y Y Y
Nelson/Marl 2010 Y Y 2 5 1 3 3 Y Y Y Y
Tairawhiti
(RTC) 2011 Y Y 1 4 4 4 Y Y Y
Taranaki (RTC) 2011 Y Y 1 1 Y Y Y
Southern 2011 Y Y 5 11 3 6 3 Y Y Y Y Y
Waitemata
(RTC) 2012 Y Y 2 3 1 2 Y Y Y Y Y
Hutt (RTC) 2013 Y Y 1 1 3
Sth Canty
(RTC) 2013 Y Y
Whanganui
(RTC) 2013 Y Y 2 2 1
Auckland
(RTC) 2014 Y
Total 11 13 17 51 19 33 18 4 8 9 8 5
*RTC refers to those DHBs that have also implemented the productive ward series ‘Releasing Time to Care’.
Final CCDM Evaluation Report (January 2015) 51
As can be seen from Figure 12, over the 12 months of progress monitoring since then and
up to the point of this final report, the roll-out has been slow. The evaluation progress
included the site visits that occurred between October 2013 and February 2014, and this
may have stimulated some activity, but it was more likely that the easier and most
progressed plans were actioned during this time.
Figure 12. Implementation of the CCDM Programme over all participating DHBs
1112 12 12 12 12 12 12 12 12 12 12
1112 12 12 12 12
13 13 13 13 13 13
38 38
4243
46
49 4950
51 51 51 51
11 11
15 15 15 15 1516
19 19 19
26 26 26 26
30 30 3031 31 31
33 33
11 11
1415 15 15 15 15
1718 18
4 4 4 4 4 4 4 4 4 4 4 4
8 8 8 8 8 8 8 8 8 8 8 87 7 7 7 7 7
9 9 9 9 9 9
7 7 7 7 7 78 8 8 8 8 8
5 5 5 5 5 5 5 5 5 5 5 5
0
10
20
30
40
50
60
Oct-13 Nov-13 Dec-13 Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14
Progress with CCDM implementation
DHBs implemented
Discovery report
Mix & Match Part 1
Wards with changes 1
Mix & Match Part 2
Wards with changes 2
Core data set
Variance response strategy
Capacity at a glance
Variance response Indicator Board
integrated Ops Centre
Table 11 below more clearly identifies the components of the programme that are slower
to be implemented. Discussion further on in the document centres on why there are
delays, and how the process may be expedited.
Table 11 also identifies who is responsible for each component of the implementation
with the DHB wide activities more likely to be completed, and the specific ward focused
activities less likely to progress rapidly.
Table 11. Implementation of CCDM Programme components (September 2014)
Activities in all DHBs implementing CCDM Proportion of
current 12 DHBs
Responsibility
Validated patient acuity tool implementation 100% DHB
Reliable validated patient acuity data 100% DHB
Negotiations & submission of EOI 100% DHB/Union
Planning CCDM implementation 100% SSHW/DHB/Union
CCDM 3 hour ‘Start up Workshop’ 100% SSHW
Discovery process 100% SSHW
Recruitment of site co-ordinator 100% DHB
Establish Organisational CCDM Council 100% DHB/Union
CCDM plan agreed & Agreement Letter signed 100% DHB/Union
Final CCDM Evaluation Report (January 2015) 52
Table 11. (Cont.) Implementation of CCDM Programme components (September 2014)
Mix and Match Part 1 (Ward work analysis) *Proportion all
wards.
Baseline measurement 60% All
Local data council established 20% All
Action plan & agreed changes made 22.40% DHB
Mix and Match Part 2 (Ward FTE calculations) *Proportion of all
wards. DHB/Union
FTE calculation & recommendations 38.80% DHB
Made changes based on recommendations 21.20% DHB
Variance Response Management Proportion of
DHBs
Capacity at a glance screen 75% DHB
Integrated operations centre 41.70% DHB
Variance indicator scoring 66.70% DHB
Standard operating response 66.70% DHB
Core data set established 30.30% DHB/SSHWU
*Calculation based on all wards the DHBs collectively planned to implement CCDM into.
The NHS Report on the scale and pace of the roll-out in the UK of the Productive Ward
programme (2010) and the Moore and Blick (2013) report on the roll-out of this
programme in New Zealand both identify the challenge of sustaining ‘whole of system’
programme implementation in a number of organisations over time. Moore and Blick
(2013) found that DHBs achieving more success with the roll-out of that programme had
the common features of:
Stronger leadership at all levels
A more structured roll-out plan
More flexibility with use of training resources.
The NHS Evaluation (2010) identified barriers to spread as:
Lack of awareness of the programme
Lack of adequate resources to roll it out
Lack of credibility or knowledge of the person/people involved and leading the
programme roll out
No clear reason given for the programme’s implementation
Language used is not well understood or jargonistic
Not enough ‘face to face’ exchanges of knowledge about the programme and
how it works (2010:29).
Further, Moore and Blick (2013), in their assessment of the roll-out of the Productive Ward
programme in New Zealand, also identified the need for the programme to be clearly
integrated with other current initiatives and projects in which the DHB is engaged.
At the time of this evaluation, the DHBs also were or had been involved in the Productive
Ward Series roll-outs and with the implementation and integration of TrendCare® with
their other software and reporting systems. There were also a number of other activities
in wards focusing on lean processes and related training that could have been
integrated more overtly with CCDM Programme activities. These included in-house ‘lean’
and change management training, HWNZ funded postgraduate nursing education
papers, and DHB quality initiatives. Integration of some of these activities at ward level
may have spread ownership of the CCDM activities.
Final CCDM Evaluation Report (January 2015) 53
Evaluation Findings
The evaluators found that only one DHB had almost fully rolled out the CCDM
programme in all wards of its hospital. Recognised in the Interim Evaluation Report were
key issues that seemed to be associated with a delay in hospital wide roll-out.
To implement the programme, the DHBs had to go through the initial nine activities
identified in table 12 above. This process was more clearly driven by the SSHW Unit
consultant(s), in partnership with a DHB Executive team advocate, mostly the Director of
Nursing.
There were parts of the programme that seemed more attractive and/or easier to
implement from the DHB perspective, particularly components of the Variance Response
Management, including the Capacity at a Glance screen (CaaG) superimposed on the
pre-existing RTC patient status at a glance screen, the variance indicator scoring, and
the standard operating response. These components were seen as enabling
management of variance in a standardised way to keep patients and staff safe in the
moment; but from a whole of system perspective, they were merely muting the impact
of mismatched staffing at the time, rather than dealing with the underlying issues of why
there was the mismatch in the first place. Dealing with these underlying issues is the
function of the two Mix and Match diagnostic tools: workload analysis and FTE
calculations.
The evaluators have discussed with both the SSHW Unit and the SSHW Governance
Group the potential for the Unit to become more of a resource for the DHBs, particularly
regarding the workload analysis and FTE calculations. The consultants now have
expertise in the programme as a whole and the specifics of how different DHBs have
managed to overcome such challenges as roster reengineering and the introduction of
skill mix. Useful strategies may include:
Expediting the workload analysis through the use of electronic technology and
having a SSHW Unit consultant support the process and produce the report.
Networking the charge nurses with others in similar settings in other DHBs for support
in implementing the ward level changes following the workload analysis
recommendations.
Completing a hospital wide FTE analysis earlier in the CCDM implementation process
to support a more strategic approach for the roll out ward by ward.
CCDM Programme Outcomes
In this section, evidence of programme outcomes obtained during the evaluation is
presented and critically examined, with reference to the recommendations of the SSHW
Committee of Inquiry’s Report.
Overall, seven key outcomes were considered by the evaluators to be directly related to
the CCDM programme:
1. Development of a pioneering workforce methodology
2. Enabling cultural change
3. Fostering and maintaining a greater level of DHB and union partnership
4. Achievement of workload variance reduction and workload smoothing
5. Standardised and “New Zealandised” NHPPD with an associated financial impact
6. Improved data integrity
7. The provision of a platform for potential for growth and development.
Final CCDM Evaluation Report (January 2015) 54
Figure 13. Key Evaluation Outcomes Categorised
Evaluation Limitations
It is important to acknowledge the limitations and confounding factors that influence the
potential of arriving at definitive findings concerning outcomes from this 12 month
evaluation. It is vital that the reader takes cognisance of the complex, adaptive nature
of the sector in which this programme was continuing to be developed as the evaluation
continued. Also, no DHBs have completed the roll-out and each was at a different level
of implementation. The key outcomes we describe within this section are those that we
believe can be isolated as attributable to CCDM. While there were a number of other
indicators that could have been analysed, the majority had too many confounding
factors to be able to accurately attribute change to the CCDM programme alone.
1. CCDM as a pioneering workforce methodology
CCDM is an innovative workforce methodology for the New Zealand health sector. Using
acuity based workload analysis appears to be a more accepted way of ensuring
workforce capacity meets service demand than the somewhat blunt instrument of ratios
(a set number of patients per nurse per shift) (Brennan & Daly, 2009; Duffield et al., 2010;
Kane et al., 2007; Plummer, 2005).
For this reason, it is important to point out that although the CCDM methodology is still
evolving and has room to grow and improve, it still appears to be a significantly more
progressive paradigm and programme than anything else currently available in New
Zealand, and apparently overseas as well (Lawless, 2014). It would therefore seem wise
to enable the programme to be fully implemented throughout the sector.
This will likely foster expansion to encompass other professional groupings and the wider
associated health workforce, so as to provide a truly whole of system workforce
management tool. Acknowledgement is made of the current widening of programme
scope to midwifery and allied/technical workforces; however, the core programme is still
predominantly nursing focused and as such is not yet truly multidisciplinary.
2. CCDM as an enabler of cultural change
One of the key attributes of CCDM, and also undoubtedly its main benefit, is the culture
change that it enables to occur at both ward and DHB level. CCDM provides the
impetus for some of the longstanding and ingrained ways of running a ward and
Final CCDM Evaluation Report (January 2015) 55
managing a hospital to be challenged. This workforce engagement and challenging of
previous ways of working is the key to the overall success of the programme.
Qualitative feedback at all levels (executive, DHB and ward) during this evaluation
highlighted the positive culture changes that have occurred following the introduction of
the CCDM programme within DHB environments. The most significant changes appear to
have occurred at DHB and ward level, with improved organisational transparency and
correspondingly increased organisational awareness by staff. The central culture change
has been a move from an often insular ‘ward view’ to a wider ‘whole of organisation’
view. The CaaG / HaaG screens have often been the greatest enablers of this change
through their visual representation of the wider system. Transparency of data in the
organisation, through the CaaG screens and the visible presentation of the Core Data
Set metrics, was noted to be a significant contributor to a culture shift.
A strong theme to emerge from the evaluation was the improvement to organisation-
wide communication, due to the transparency of the data. It was signalled to the
evaluators that communication had improved throughout the organisations. This has
helped to break down silos as wards deploy staff to assist with negative variation, and
this is reciprocated when required. This in turn has meant that ward and DHB staff have
greater insight into the wider organisation, and the culture has begun to reflect this as a
result.
The impact of this culture change on the wider DHB and ward environment cannot be
overstated, as it is one of the key benefits of this programme. It could also be argued
that this type of culture change is very difficult to implement and lead within large and
hierarchal institutions. This level of change has not always occurred with many other
large-scale sector change initiatives, and in this regard, the processes imbedded within
CCDM have been able to achieve considerable gains in relation to change
management.
Environments that appeared to have had the most progress in cultural change were
those where the ownership and interest in the CCDM process (to achieve a healthy
workplace for staff) were maintained and championed at executive and middle
management level. The programme seemed to flourish most when the Executive drove
CCDM and were fully cognisant of the wider process, rather than merely acknowledging
it as one of many tools that could be used to help manage a DHB system and better
utilise the existing workforce. Indeed, success in implementation at hospital level
appeared also to be heavily dependent on strong and well-connected clinical
leadership, particularly in nursing. Having a DON who was committed to the programme
appeared pivotal to its success.
At ward level, the charge nurse needed to have the skills and confidence to implement,
manage and maintain change. Qualitative feedback gained in the evaluation also
identified that having a Charge Nurse or team leader committed to the programme was
another key component to the success of CCDM implementation at ward level.
3. Fostering and maintaining a greater level of DHB and union partnership
A fundamental aspect to CCDM as a programme is the level of union engagement, and
indeed union partnership, which underpins the premise of the entire programme. This
level of engagement is often not seen in other large-scale sector change programmes
associated with workforce change. The embedding of this partnership process has been
critical to CCDM’s roll-out, and is what sets it apart from other potential staffing solutions
such as mandated ratios. The original SSHW COI Report (2006) called for a whole of
sector response and a different approach from previous attempts at finding a
sustainable and inclusive solution to workforce issues. This is precisely what differentiates
this approach from previous sector initiatives; the fact that CCDM is a process by which
workforce leads ER/IR. By contrast, the majority of the current health workforce
environment operates in the opposite way, with the ER/IR environment leading
workforce.
Final CCDM Evaluation Report (January 2015) 56
The fundamental approach, unique to CCDM, is that workforce engagement by all
parties, union and employer, is of equal standing, and that it is not only at the bargaining
table at which these conversations occur: the unions are represented on the central and
local data councils, and are involved in the process as it rolls out.
The level of effectiveness of this partnership has varied throughout the DHBs included in
the evaluation, with some DHBs having obvious issues between both parties, and
subsequently less effective CCDM roll-out than others. However, where the partnership
has worked well and trust has been built up on both sides, this appears to assist
substantially with overall CCDM implementation success.
Overall, the value of the union/employer partnership cannot be overstated and should
be deemed absolutely critical to CCDM roll-out and also to on-going CCDM success for
a DHB.
4. Achievement of variance reduction and workload smoothing
The SSHW COI Report identified a series of sequential ‘steps to achieving a safe
workplace for nurses and midwives (2006, p66). The following outcomes are presented
within the framework of these steps.
4.1 Forecasting patients (SSHW COI Step 1)
Described as a vital first step, forecasting includes obtaining detailed information on both
elective and acute clients expected, their projected length of stay and the nature of
care required. According to the SSHW Unit, a DHB must agree on the data that is
needed to forecast demand, using retrospective, as well as, prospective data.
The CCDM programme recommends that DHBs use their District Annual Plan (DAP)
process to forecast their long-range demand (1-3 years), taking into consideration
seasonal and artificial variance. Forecasting the medium range CCDM (3-6 months)
requires scanning data from targets and operational norms obtained from the shared
data set. Forecasting CCDM short range (1-6 weeks) is achieved through development
of an organisational dashboard and operations centre activities.
Tools and activities developed within the CCDM Programme to achieve this:
The use of the validated acuity tool TrendCare® provides retrospective data.
An expert forecasting advisory group has been developed to help progress work in
this area.
Evaluation Findings
Forecasting has proved to be difficult for some DHBs, as the CCDM programme does not
have a specific tool that provides this prospective data nor a methodology to collect it.
In response, the SSHW unit has established a mid-term forecasting group, which sits
outside the CCDM programme. The group is currently looking at the best practice
literature, and eight pilot DHBs are completing a survey questionnaire on forecasting.
They acknowledge that there has been a gap in forecasting.
TrendCare®, the validated acuity tool used in the CCDM programme, can only forecast
based on previous patterns, and is reliant on continuation of a regular pattern of patient
demand. Currently DHBs are either using specific forecasting software, or the DHB’s
Decision Support unit is driving forecasting in house.
According to the SSHW COI Report (2006, p. 52), ‘quality and accuracy of this forecast,
and the extent to which it is used as the basis for planning and staffing, are the most
significant factors in achieving a safe and healthy match between patients and nurses or
midwives’. It can be argued that the CCDM programme has not met this step, and this is
an area for further improvement. However, other activities undertaken by the SSHW Unit
are working towards achieving consistency with forecasting for DHBs.
Final CCDM Evaluation Report (January 2015) 57
4.2 Smoothing the planned workload (SSHW COI Step 2)
This step involves removing as much variance as possible from the patient forecast
against which the staffing roster has been developed.
Tools and activities developed within the CCDM Programme to achieve this:
Requiring frequent and regular TrendCare® actualisation
Workload analysis (Mix and Match Part 1)
FTE calculation (Mix and Match Part 2)
Variance Response Management tools
Nursing Productivity Pre and Post CCDM – DHB Exemplar
Some DHBs have reported better matching of capacity to demand after CCDM
implementation. While there will be other confounding factors that can affect this
outcome, in one DHB the evaluation was able to further explore workforce differences
pre and post CCDM, utilising DHB data to map to a productivity line. The graphs below
show AM, PM and night variation pre and post CCDM for a medium sized DHB. The red
line demonstrates alignment to overall productivity (100% productivity being an absolute
match of capacity to demand).
The AM shifts demonstrate changes pre and post CCDM, with the PM and night shifts
being less conclusive. However, the evaluation has sought to highlight all findings, not
only those that demonstrate clear changes. The fact that there have been notable
differences in AM shift productivity ratios and not in PM and night shift is an interesting
finding.
AM Shifts
For AM shifts, the majority of the post 11/12 CCDM data (the green line) are a better
match than the pre 09/10 CCDM data (blue line), barring the spike in March. However,
this spike readjusted and was a closer match by May/June. The AM shift highlights best
the improvements seen by this DHB pre and post CCDM implementation.
In the pre CCDM data, apart from July/August, the 2009/2010 ratio is never particularly
good for the morning shifts. The pm shifts show a post CCDM improvement, but this is,
interestingly, not as significant as the AM improvement. The night graph does not
demonstrate any significant findings pre and post CCDM implementation. This is a
common pattern across all DHBs, reflecting minimum staffing numbers on night shifts.
Figures 14, 15 and 16 show the graphs for the three shifts.
Figure 14. AM Shifts
Final CCDM Evaluation Report (January 2015) 58
PM Shifts
Figure 15. PM Shifts
Night Shifts
As noted above, the night graph does not demonstrate any significant findings pre and
post CCDM implementation. However, it does raise the question of why there are
notable differences in the AM graph, smaller changes in the PM graph and no notable
changes in the night graph.
Figure 16. Night Shifts
4.3 Estimating patient and non-patient generated staffing (SSHW COI Step 3)
This step involves the use of a tool to generate the basic staffing plan, based on actual
patient need at the time. This plan should be designed by the service/ward to meet its
unique patient needs, and then continue to be managed by the nurse/midwife
manager at that level. The plan also needs to include time for non-patient quality
assurance activity.
Final CCDM Evaluation Report (January 2015) 59
TrendCare® is the tool used by CCDM for this purpose, and it is tailored for each ward
following the workload and FTE analysis. The actualisation of TrendCare® by the
nurse/midwife providing care to the patients forms the vital function of checking that the
staffing at the time and for the following 24 hours will most closely meet the patient care
needs. Data integrity requirements suggest aiming for 100% actualisation rates. If
patients’ acuity is not actualized, the hours captured for reporting on patient acuity may
be incorrect and data integrity compromised.
Tools and activities developed within the CCDM Programme to achieve this:
Data generated out of TrendCare®
Workload analysis (Mix and Match Part 1)
FTE calculations (Mix and Match Part 2)
Local data council
Evaluation Findings
Based on the pre and post CCDM actualisation rates, it would appear that (excluding
one outlying DHB) the actualisation of TrendCare® is more likely to increase following the
introduction of CCDM. All participating hospitals are required to have TrendCare®
installed and being used for at least 6 - 12 months prior to CCDM implementation.
However, with the process of CCDM implementation and heavy focus on reports
generated out of the product, actualising clearly takes on a greater value. Figure 16
compares TrendCare® actualisation rates pre and post CCDM implementation. We note
that none of the DHBs included in this analysis achieved 100% actualization rates.
However, this evaluation concludes that the CCDM programme acts to improve data
integrity with increased actualisation rates post implementation, which in turn enhances
the DHB’s return on investment in TrendCare®.
Figure 17. Actualisation rates across DHBs pre and post CCDM*
*data was only available from 5 DHBs in time for this report and the DHB with lower post CCDM actualisation
rates (B) had reduced resources into TrendCare® & CCDM support by the time the evaluation took place.
DHB Variance Pre and Post CCDM Findings
Data collected from participant DHB sites provided pre and post CCDM NHPPD.
Differences in NHPPD highlighted the variances after implementing CCDM within a DHB.
The following graphs outline this variance in NHPPD over the three shift types; AM, PM
and night. The first demonstrates the under capacity on day shifts and over capacity on
afternoon and night shifts in most DHBs.
Final CCDM Evaluation Report (January 2015) 60
Post CCDM, there is a noticeable reduction in the degree of variance in evening and
night shifts. The final graph (Figure 21) demonstrates the aggregate variance and
changes pre and post CCDM. The CCDM tools can inform the redistribution of staffing
over the 24 hour day to better meet patient demand.
AM Shift Variance
This graph demonstrates the staffing under capacity (too lean) occurring in four of the six
DHBs pre CCDM on the AM shift. For the majority of DHBs, this had changed very little
post CCDM. One DHB, however, had considerably worse under capacity post CCDM.
One DHB had substantial over capacity both pre and post CCDM; however, post CCDM
this had reduced considerably, so the variance was significantly smaller than pre
implementation.
Figure 18. NHPPD Variance Pre and Post CCDM - AM Shift
Figure 19 demonstrates the changes in variance pre and post CCDM implementation for
the PM shifts. In almost all DHBs the pre CCDM variance is significantly higher than post
CCDM, demonstrating that over capacity variance has reduced following the
introduction and roll-out of CCDM.
Figure 19. NHPPD Variance Pre and Post CCDM – PM Shift
Final CCDM Evaluation Report (January 2015) 61
Figure 20 highlights the overcapacity on night shifts occurring at all DHBs both pre and
post CCDM. However, post CCDM there is a reduction in overcapacity variance in
almost all DHBs, except one where it becomes greater.
Figure 20. NHPPD Variance Pre and Post CCDM - Night Shift
Figure 21 highlights the aggregate of variance for all DHBs, according to shift type.
Interestingly, the morning shift shows that there is consistent under capacity both pre and
post CCDM; however, in both the evening and night shifts, while there is still over
capacity post CCDM, the variance has been reduced. But in the AM shifts, the under
capacity has increased overall.
Figure 21. NHPPD Variance Pre and Post CCDM – Aggregate All DHBs per Shift Type
Final CCDM Evaluation Report (January 2015) 62
Evaluation Findings
The above graphs represent the first inter DHB benchmarking of CCDM generated tools
and activities. This provides an opportunity to identify ‘outliers’ and explore the reasons
for the differences. It also provides an opportunity to identify consistent/across DHB
trends, for example the AM shift under capacity (figure 18), present pre-CCDM in 4 of the
DHBs, appears to have become worse in 2 DHBs and not changed in another post-
CCDM. The value of CCDM is the ability to be able to benchmark, take action and then
measure the impact in a standardised open and transparent way.
4.4 Estimating the effect of other moderating factors (SSHW COI Step 4)
This step identifies the influence of the cultural environment, including leadership,
authority and teamwork, as well as the physical environment of the workplace, on the
health and well-being of staff.
Tools and activities developed within the CCDM Programme to achieve this:
Discovery process
DHB CCDM Council
Workload analysis (Mix and Match Part 1)
Local Data Council
Evaluation Findings
Recognising that quantifiable measures of this step will be difficult to achieve, there are
some activities of the CCDM programme that demonstrate cognisance of this step. They
are the integration of RTC activities into the CCDM Programme at ward level and the
role played by the Local Data Council.
All but one of the active CCDM DHBs have also implemented Productive Ward:
Releasing Time to Care (RTC). This uses a lean approach to maximise the efficiency of
care. The CCDM Programme has built on the activities of the RTC initiative through use of
the ‘Knowing How We Are Doing’ process as a basis for development of the Core Data
Set, recognising the efficiencies gained through the Well-Organised Ward component.
With regard to staff health and safety measures, these have been more difficult to
measure and attribute to CCDM. These issues are discussed further on in the report.
5. Direct impact on NHPPD and associated financial impact
5.1 Provision for leave (SSHW COI Step 5)
Calculation, inclusion and planning for all entitled leave types are included in the FTE
calculation.
Tools and activities developed within the CCDM Programme to achieve this:
FTE calculation (Mix and Match part 2)
Evaluation Findings
The FTE calculations included all leave types as they related to each staff member when
working though the analysis, which was then designed to be used as the base staffing
roster.
5.2 Fine tuning and budgeting (SSHW COI Step 6)
Once the staffing requirements have been matched to the forecast workload, this
should form the basis of budgeting decisions. The CCDM programme uses the FTE
calculation tool as the basis for budgeting. Investment in getting the staff base right has
a high cost/benefit ratio, as it maximizes service delivery and minimises wasting
resources.
Final CCDM Evaluation Report (January 2015) 63
Tools and activities developed within the CCDM Programme to achieve this:
Regular and frequent TrendCare® actualisation
Workload analysis (Mix and Match Part 1)
FTE calculations (Mix and Match Part 2)
FTE calculation (Mix and Match Part 2) reports – three examples
There have been 31 completed FTE calculation (Mix and Match Part 2) reports in nine
DHBs. However, it was signalled to the evaluation team that in many instances, these
reports had not been completed or changes implemented, due to budgetary concerns.
This is supported quantitatively, as roster re-engineering and changes had occurred in a
total of only 14 wards at the time of writing. Two wards were identified to have made
adjustments to the method after discussions with their finance team, in order to make
adjustments to FTE that were in line with the organisation’s budget setting.
The evaluation team requested examples from DHBs of these reports from wards in which
FTE changes had been implemented as a result of the process. The purpose of these
examples was to examine the financial implications for a DHB implementing
recommended increases in nursing FTE.
Recommended changes were not strictly related to an increase in nursing FTE. For
example, two wards moved to a different model of care, and one ward introduced
Health Care Assistants to the ward previously using an RN only model of care. Prior to
implementing the recommended increase in FTE, there was evidence that patient safety
was at risk in two of the wards with a persistent gap in nursing hours supplied versus time
required by patients on AM and PM shifts, and the staffing picture pre-implementation
showed evidence of care rationing.
Table 12 outlines the variance between ward budgeted FTE pre CCDM and post CCDM
FTE calculation, with additional comment on the implementation.
Table 12. Case study findings of implementation of FTE increases in 3 wards
Ward Current
Ward
Budget
FTE
Calculation
Variance
from initial
to
calculated
budget
Comment on implementation
1 22.43
FTE
26.82 FTE 4.39 FTE Equivalent FTE was already being spent
on casual utilisation. Post
implementation ward casual utilisation
average costs per month decreased by
two-thirds.
2 22.99
FTE
26.22 FTE 3.23 FTE FTE was reconfigured across two wards
and was implemented as cost neutral.
The ward has since maintained budget.
3 34.5 FTE 43.32 FTE 8.82 FTE Pre CCDM the ward was significantly
over budget. The actual ward spend
was equivalent to 41.54 FTE. Equivalent
FTE was being spent on casual utilisation
and existing staff flexing up in an ad
hoc manner. The ward has now
maintained budget post
implementation and the roster is stable.
Final CCDM Evaluation Report (January 2015) 64
These case studies provide valuable insight into the resource actually required to
increase the base staffing to a level that will reduce reliance on casual staff. Two wards
were already utilising the equivalent FTE in ward casual utilisation and flexing up existing
staff ad hoc. One ward was able to implement the FTE as cost neutral due to an FTE
reconfiguration. Although limited, these examples provide evidence that it is possible
that a DHB is already utilising the FTE required in more expensive, more unstable and less
acceptable ways for existing staff.
Evaluation Findings
The most measurable financial benefit relates to savings in nursing hours by getting the
base right and responding to variance. This was achieved in DHBs, at the ward level, with
agreed staffing variance processes and plans, actively responding to variance and
discussing variance at daily integrated operations centre meetings.
Analysis of HPPD aggregate year end data indicated that:
- As shown in Figure 18, 86% of participating showcase wards experienced negative
variation in NHPPD during the AM shift pre CCDM, compared with 71% of showcase
wards experiencing negative variation in NHPPD over the year end post CCDM.
- Figure 19 shows that during the PM shift, 43% of showcase wards experienced
negative variation post CCDM compared with 29% pre CCDM.
- Figure 20 outlines pre and post CCDM implementation. 100% of the wards
experienced positive variation in NHPPD during the night shift, with many wards able
to reduce significant overcapacity.
- The post implementation year end NHHPD data displays an overall reduction in the
shifts experiencing negative variation. A caution needs to be sounded over the risk of
running staffing too lean.
Benefits in Nursing Hours
Examples of financial benefits attributed to a reduction of variance in nursing hours
based on $36.50 per hour ($27.50 based on collective agreement plus additional DHB
employment costs) are outlined in Table 14. Savings from these wards and aggregated
data are comparable with DHB savings identified in 2012-13 of 21,183 nursing hours; at a
conservative rate of $36.50 per hour, this comes to a total of $773, 180.00.
This model is based on an average ward of 25 beds, a DHB with a modest 200 beds, and
a modelling timeframe of five years, discounted to present day value. This is assumed to
be sufficient time for a DHB to implement at least some of the CCDM tools, especially
VRM. It must be acknowledged that once changes have been implemented at both a
ward level and DHB wide, the five year model would not show the same exponential
benefits year on year. This figure does, however, represent savings that would not be
made if the DHB had not acted to respond to variation and additional recommended
changes to respond to variance.
The financial benefits of getting the base right and responding to variance can be seen
for DHB F in Table 13. This DHB has implemented CCDM organisation wide and is close to
being ‘over the line’. It is an example of what is possible with full CCDM implementation.
This includes significant cost savings attributed to reducing variance. The ward data
analysed from DHB F has seen a significant year end reduction of 9658.91 NHPPD. This
reduction is attributed to reducing the positive variance during the PM and night shifts.
Caution is warranted in ensuring that the DHB has not reduced staffing too lean on the
day shift, as this may result in unsafe staffing. DHB E is an example of a DHB which has
implemented some of the CCDM suite of tools and is progressing well, but still has some
additional work to be done to get ‘over the line’. The ward in DHB E also experienced a
reduction in variance NHPPD, attributed to reducing positive variation in NHPPD during
the PM and night shifts. This is demonstrated in Figures 20 and 21 above.
Final CCDM Evaluation Report (January 2015) 65
The ward has also reduced some negative variation during the day shift. It must be
acknowledged that a limitation of using year end NHPPD is that it is crude and does not
identify individual shifts with negative variance, which international literature associates
with an unsafe working environment.
Table 13. Ward level Financial Benefit data at ward level, DHB level and over 5 years
Showcase
Ward
Benefit
NHPPD
Ward level
(25 beds)
(NHPPD x
$36.50)
DHB Level
(200 beds)
(Ward level
calculation x 8)
DHB Benefit
over 5 years
(discounted)
(DHB level
calculation x 5)
Ward in DHB F 9,658.91
$352,550.22 $2,820,401. 72 $14,102, 008.60
Ward in DHB E 1,552.71 $56,673. 92 $453,391. 32 $2,266,956. 60
Ministry of Health Financial Impact Analysis
During the course of this evaluation and independently from it, the Ministry of Health
undertook an analysis of the financial impact of the CCDM programme on selected
participating DHBs. This analysis included examining the patterns of variation in a range
of financial, human resource and efficiency measures in 10 cost centres in three DHBs .
The measures were analysed before and after the introduction of the CCDM
Programme. Control charts and statistical patterns of ‘special cause’9 variation were
used to identify changes in the measures attributable to the Programme.
This analysis concluded that:
Net expenditure per bed day stayed the same in eight cost centres, increased in one
and decreased in one. The financial impact of FTE changes associated with the
programme, as measured by net expenditure per bed day, is difficult to isolate from
other factors that drive changes in patient volumes, such as seasonal variations
(winter peaks; summer holiday dips), ambulatory models of care, bed or theatre
closures and service reconfigurations.
Total nursing labour costs increase where the programme shows more nurses are
needed to meet the care demands of patients, and the nursing resource had not
previously been supplied. Analysis showed that total nursing labour costs increased in
three of the 10 cost centres as a result of implementing recommended increases in
Full Time Equivalents (FTE). This demonstrates the use of the programme tools,
including the validated acuity system, in identifying the actual care demands and
workforce requirements.
Matching nursing resource to patient need improved over time in four cost centres
studied. Failure to achieve an improved match in the other cost centres is partly
attributable to poor integration with other parts of the system and poor midterm
forecasting and planning - for example, where ward or bed closures and operating
list downtime have not been planned in conjunction with the nursing resource.
Improved performance was most evident in one DHB where reduced ‘special cause’
variation was found in almost all measures in the July 2013 to June 2014 financial
year. Reduced variation is widely regarded as the foundation to quality
improvement. This provides compelling evidence of an effective whole of hospital
approach to the programme.
9 Special causes are those causes not part of the process all the time or that do not affect everyone, but arise
because of specific circumstances e.g. a surgeon going on leave or a change in model of care.
Final CCDM Evaluation Report (January 2015) 66
This analysis concluded that there are some costs associated with increasing nursing
resources where, based on the evidence, this resource is needed to provide safe staffing
and quality care for patients. However, there is potential for these costs to show
downstream financial benefit where whole of hospital planning is achieved, and where
improvement in quality is able to be measured (MoH, 201410).
5.3 On the day (SSHW COI Step 7)
This step involves ensuring that there are mechanisms in place to make sure that on the
day, the nurse/midwife in charge has the ability to ensure that the right person with the
right skill set is available to provide care for the patient.
Tools and activities developed within the CCDM Programme to achieve this:
Regular and frequent TrendCare® actualisation
Workload analysis (Mix and Match Part 1)
FTE calculation (Mix and Match Part 2)
Variance Response Management
Workload analysis (Mix and Match Part 1) - Deployment Data Exemplar
A hospital ward established a PM shift HealthCare Assistant role to manage
environmental and administrative activity. The need for this role became evident from
the workload analysis data, which identified the volume of activity and timing of activity
to be high during the PM shift, yet there was no HCA working on that duty. The PM shift
was the only shift in which a change was implemented as a result of the workload
analysis. This enabled the RNs to focus on patient care.
An analysis of variance pre and post CCDM implementation indicated that there was
most often under capacity during the day and over capacity at night. This is in line with
the data outlining the deployments into the ward; the majority of deployments occur
during the day, whereas only 14% of all deployments occur on the night shift.
Further analysis of the deployment data for a year pre and post CCDM implementation
indicated a 7% drop in deployments into the ward in the PM shift, where the additional
resource had been allocated. It is likely that this additional FTE resource added to the PM
shift has decreased deployments into the ward during that shift.
Pre and post CCDM implementation - Ward Variance Reporting Exemplar
To further demonstrate CCDM impact pre and post implementation, we have provided
an exemplar from a medium sized DHB. We have taken the ward variance report and
compared pre and post implementation figures for a specific ward. Figure 22
demonstrates what the averages per shift look like across the year (pre and post). The
green and purple bars show post implementation figures.
10 Communication from MOH Chief Nurse (December 2014).
Final CCDM Evaluation Report (January 2015) 67
Figure 22 HPPD pre and post CCDM
Day Evening Night
2009/2010 Avg HPPD Required
Pre CCDM2.04 1.64 1.05
2009/2010 Avg HPPD Provided
Pre CCDM1.82 1.86 1.15
2011/2012 Avg HPPD Required
Post CCDM2.08 1.74 1.03
2011/2012 Avg HPPD Provided
Post CCDM1.95 1.82 1.19
0
0.5
1
1.5
2
2.5
HPPD
Evaluation Findings
Figure 22 demonstrates that there has been a definite improvement on the AM and PM
shifts. For example, the difference between required and actual HPPD on the AM shifts
for 2009/10 is 0.22 HPPD; for 2011/12 it is 0.13 HPPD. This is an improvement of 0.09 HPPD
(0.05 of an hour). If we calculated out these minutes per patient over this period of time,
even this small amount of time would still add up and should be noted as an
improvement. Findings for the PM are similar, except that this involves a decrease in
oversupply of hours. Unfortunately there was no improvement for the night shift, but
rather a slightly worse situation than in 2011/12. This situation will be difficult to resolve,
owing to the difficulties in achieving economies of scale on nights, where there needs to
be a minimum staffing presence.
6. Improved ability for a DHB to harness its acuity data
A significant benefit of the CCDM programme is the ability for DHBs to substantially
improve their current acuity data, in terms of both collection of data and ability to use
data to better inform decision making within the wider institution. The CCDM pre requisite
of a reliable period of accurate validated patient acuity data can help inform this
process further and add to future data quality. Some DHBs have forecasting ability, but
not a validated patient acuity tool. During the course of the evaluation, it has become
increasingly clear that this can be a large disadvantage, as without such a tool there is
no way to accurately match capacity with demand. So one of the main strengths of
CCDM is that it offers DHBs the ability to start reliably and proactively using acuity data to
help inform organisational decision making. Well utilised acuity data assists at all levels,
ward, DHB and national, as it enables improved planning and implementation overall.
Final CCDM Evaluation Report (January 2015) 68
6.1 Incident responsiveness (SSHW COI Step 8)
This step requires the presence of a detailed and workable plan to manage the ability to
respond to increase in demand over capacity of staff to respond.
Tools and activities developed within the CCDM Programme to achieve this:
Variance Response Management
Variance Indicator Boards
Evaluation Findings
The level of responsiveness throughout the DHBs in terms of this step is mixed. While the
majority of DHBs have implemented VRM, what this means at ward level when demand
exceeds capacity is not always clear. Most DHBs have outlined plans for what staff and
wards should do when this situation arises; however, the degree to which these plans are
both fit for purpose and executed is often not consistent. Where VRM has been
embedded hospital wide, there is more opportunity for plans to be able to be fulfilled
and for redeployment to occur. This is particularly true for DHBs which reinforce
redeployment, and which actively encourage a wider ‘whole of hospital’ view of staffing
to be taken. There are definite gains to be made in this area with wider implementation
and programme embedding.
6.2 Review (SSHW COI Step 9)
Review and fine tuning of the staffing plan need to be ongoing. The regular review of
patient forecasts also forms an important part of this process.
Tools and activities developed within the CCDM Programme to achieve this:
DHB and Local Data councils
Regular and frequent TrendCare® actualisation
Annual FTE calculation (Mix and Match part 2)
Core Data Set analysis
Evaluation Findings
As for Step 8, the implementation across the DHB sites with regard to regular review is
mixed. However, for DHBs which have more fully implemented CCDM, regular review
can be seen to be more embedded and occurring as part of DHB and local council
meetings, as well as annual FTE calculation and forecasting for budgetary purposes.
There are varying degrees of analytical review, which occur at the council level (both
local and DHB). In part this can be due to where a DHB is in the implementation process,
as well as to how well the council understands and owns its data. The more proactive a
council, the greater likelihood there is of review actually helping shape future patient
and workforce decision-making processes.
7. Further programme potential for growth and development
The final key outcome from the evaluation is acknowledgement of the ability for further
growth and enhancement of the CCDM programme. The programme is still relatively
new in terms of establishment and implementation. There is potential to move it from
where it is now to a more developed state. The most logical first step would be to
consolidate the existing programme and implement it fully in all DHBs and all wards.
Once this has been achieved, further expansion of the programme would be possible
through a wider multidisciplinary focus and further development of mid-term forecasting.
The key point is that the programme still requires consolidation. While it has been
implemented within the majority of DHBs nationally, many DHBs have elected to roll the
programme out in a piecemeal way. Not one DHB has implemented CCDM throughout
its entire organisation. The evaluators consider that this needs to be done as the top
current priority, with further development and inclusion of other disciplines the next focus.
Final CCDM Evaluation Report (January 2015) 69
Impact of CCDM
The following section of this report identifies the actual and potential impact of the
CCDM programme to date, framed within the SSHW Committee of Inquiry Report (2006)
goals of safe staffing:
o Assuring patient safety and satisfaction
o Supporting staff health and wellbeing
o Maximising organisational efficiency.
1. Patient Safety and Satisfaction
For a number of reasons, patient safety and satisfaction indicator results could not be
attributed specifically to the CCDM programme. These reasons included:
An incomplete roll-out of the programme in any DHB.
The programme being rolled out concurrently with other whole of sector
programmes, such as Releasing Time to Care, and other productive ward and
specific patient safety programmes relating to medication management, infection
prevention and reducing harm from falls (http://www.hqsc.govt.nz/). Therefore there
are too many confounding factors to accurately attribute increased safety in these
areas specifically to CCDM.
We did explore the option of analysing data on adverse patient outcomes, but due to
the infrequency of the events and confounding factors, we decided not to use this data.
An earlier attempt had been made to do this by the Health Services Research Centre of
Victoria University (2013) in a quantitative evaluation of the Mix and Match
methodology, but the findings were inconclusive. The regression analysis results indicated
that patient perceptions of care were ambiguous and not easily attributable to a
specific service change.
The only reliable measurement that could be directly attributed to CCDM was variance
reduction. From an organisational view point, CCDM does provide a framework for
identifying safe staffing levels and reducing variation that ultimately improves patient
flow through the system. Variation is identified as being directly related to quality care
(IHI, 2014). Reducing variation improves the predictability of outcomes and reduces the
frequency of poor results.
The Variance Response Management suite of tools is designed to manage variance
within a structured, standardised and universally understood process. CCDM provides
tools to predict monitor and manage variability by providing:
an early warning system to detect and manage workforce shortages on the day
a more detailed overview of the whole hospital so staff can recognise and act on
apparent increases in patient demand
tools to manage unexpected variance and maintain patient safety.
The workload analysis and FTE calculation processes both enable improved patient
safety through more appropriate skill mix of staff, roster re-engineering, and models of
nursing care which increase the likelihood that the right staff will be available to provide
care to the patient at the right time in the right place. There is now strong evidence
indicating a direct relationship between NHPPD and patient outcomes (Kane et al., 2007;
Needleman et al., 2011). Needleman et al. (2011) identified an association between
patient mortality and increased exposure to shifts that were deficient 8 hours or more
below target staffing, and found that this increased risk was cumulative. Lankshear et al
(2005) reviewed 22 studies and found that a richer skill mix, especially of registered nurses
and higher nurse staffing, is associated with improved patient outcomes.
Final CCDM Evaluation Report (January 2015) 70
The study found that when there were lower NHPPD, there were more episodes of care
rationing, which directly affects patient outcomes. This finding would support the CCDM
programme’s efforts to factor NHPPD into nurse staffing models. Additional research has
identified a relationship between high rates of nursing staff turnover and adverse patient
outcomes, as well as inadequate staffing levels being associated with job dissatisfaction
(McGillis & Hall, 2005; SSHW COI Report, 2006).
CCDM recommends a Core Data Set which, if applied, will capture information on
patient safety and satisfaction. However, DHB capacity to capture this data in real time
is currently poorly developed, and findings will not necessarily be able to be attributed to
CCDM alone.
2. Supporting Staff Health and Wellbeing
There is some evidence pointing to the conclusion that when a DHB engages in the
CCDM programme, staff satisfaction is improved. The results from three different staff
engagement surveys and one research study were reviewed to assess the impact of
CCDM on staff wellbeing.
Analysis of sick leave showed a downward trend over the four years since one DHB had
implemented CCDM; but over the past decade, the rate had fallen more significantly
prior to CCDM implementation, so this decrease was not specifically attributable to
CCDM.
DHBs’ staff turnover data was also analysed. Although the turnover was trending toward
decreasing, it was difficult to determine if this was related to CCDM or to the economic
climate.
Analysis of absenteeism data at ward level in ‘CCDM showcase’ showed a small decline
in absenteeism with CCDM implementation, but this is not easily attributable to CCDM.
As part of the SSHW Unit’s function, they conduct pre and post CCDM implementation
staff surveys, using the e-tool Survey Monkey®. The evaluators sought results from post
evaluation surveys, but because of the delays in the roll-out in DHBs, few post CCDM
surveys had been completed, and those that had featured poor response rates,
because of the other staff surveys being undertaken within and by the DHB.
Table 14 shows the most recent available results from surveys conducted post CCDM
implementation. Interpretation of the results needs to be made with caution, as they
may not necessarily be attributable to CCDM.
Table 14. Staff satisfaction and wellbeing survey results
Measuring Staff
Satisfaction
Findings post CCDM
implementation
Comments
DHB ward level staff
satisfaction survey
Pre and post CCDM
implementation
conducted by SSHW
unit (2013) using an
e-survey tool.
Effort required to maintain the service
at current level is unchanged.
Dramatic drop in staff anxiety
associated with staffing levels.
Increase in the perception of personal
pressure associated with equitable
workload allocation and scope of
practice.
Staff signalled they felt the organisation
placed a priority on achieving volumes,
targets and budgets more than patient
care and staff wellbeing.
Staff perception in this DHB
has been driven largely by
VRM, as there had not been
any Part 2 calculations
completed at the time of
survey.
NZNO employment
survey –
Sub analysis of
responses from DHBs
post CCDM
implementation
(2013).
Nurses from DHBs, which have
implemented components of CCDM,
have a high awareness of the
programme with the exception of one
DHB.
It is likely that very few DHBs
have implemented enough
of the CCDM to create a
tipping point whereby nurses
experience a change to
their work environment.
Final CCDM Evaluation Report (January 2015) 71
38% of respondents indicated that
there were enough nurses working in
their DHB to provide safe care.
Nurses, for the most part responded
their work environment had remained
‘unchanged’ since their DHB
implemented CCDM.
An increase in the
completion of Part 2 may
create a different result as
nurses experience a change
at the coalface.
DHB-wide staff
satisfaction survey
Pre and post CCDM
implementation
conducted by SSHW
unit (2014) using an
e-survey tool.
No change signalled from respondents
in work conditions and environment,
their perceptions of their job, patient
care and staffing structures and
processes.
A 7% decline in the perception of staff
that there are sufficient staffing levels
and resource available.
The sample size was small
and the DHB had
completed only 1 FTE
calculation in one ward.
Victoria University
Health Services
Research Centre-
Quantitative
Evaluation of the Mix
and Match staffing
methodology (2013)
Staff perceptions of the work
environment, the care they provided
and reports on care rationing events
were better on shifts with adequate
staffing in accordance with CCDM
methodology.
At the time only 24% of the
734 shifts met the shift design
criteria as being safely
staffed.
Collectively, these findings do not demonstrate any significant improvement in staff
satisfaction and wellbeing as a result of CCDM. Given the incomplete roll-out, the
disruption caused by the CCDM activities at ward level and the degree of change
required in some areas, this is not surprising.
The degree of change, such as the introduction of skill mix, roster re-engineering and
new models of care, required in some areas will be very unsettling for some staff.
However, the use of TrendCare® as the validated acuity tool across all CCDM
implementation sites does provide some comfort for those implementing the change
that the FTE calculations are based on standardised and benchmarked NHPPD. Nurses in
wards that have been identified as over staffed on some shifts in the past will find
adjusting to the new roster more difficult, and may perceive that they have an increased
workload.
Conversely, at the evaluator site visits, feedback from staff who had been involved in the
workload analysis and FTE calculations was very positive about the impact of the
programme. They felt that the processes involved were transparent and inclusive. While
there were changes in how variance was managed, with more overt staff deployment,
indications were that this was becoming more acceptable and that the wards and
hospital were working more as a team. This was reinforced by the use of the CaaG
Screens so that workloads were transparent, and there was a growing culture of
assistance being offered rather than requested. Nurses stated positively that they got to
know the other parts of the hospital more and met more staff other than those they
worked with regularly. The nurses provided this feedback to the evaluators without
management present. These were consistent findings.
3. Maximising Organisational Efficiency
It would appear that CCDM has the potential to increase organisational resilience and
that indicators within the core data set can be used to benchmark progress. A resilient
organisation is one that can sustain outcomes despite a variation in conditions such as
demand, and is able to maintain safe staffing levels, which in turn achieve good
outcomes for patients, while also achieving efficiencies. Basing a Core Data Set on
metrics designed to capture quality patient care, best use of DHB resources and the
work environment would enable DHBs to develop a more focused approach to service
improvement.
Final CCDM Evaluation Report (January 2015) 72
Regular, open and transparent monitoring of the Core Data Set metrics would provide
quick and reliable feedback on the impact of change on staff, patients and the
organisation as a whole.
This evaluation observed DHBs allocating a considerable amount of time and effort, at
the CCDM council level, to developing an agreed Core Data Set. Despite the efforts of
many people within the DHBs, this evaluation identified that the Core Data Set was not
established in most DHBs visited. Many DHBs were collecting metrics ad hoc and were
able to use these metrics to inform decisions; however, as the metrics were not a part of
the collective Core Data Set, they could not enable a whole of system overview and
contribute as a whole to improving organisational efficiency.
A decision will need to be made as to whether there is to be a national level
standardized Core Data Set (with clearly defined indicators), which would enable
benchmarking. Ideally these metrics would have common definitions and be based on
current DHB reporting. Additional metrics that each DHB would like to collect at an
individual level to inform local organisational efficiency could still be beneficial to that
organisation.
A notable difference identified at site visits where the CCDM programme had been
more established was the increase in communication and transparency reported at
each level of the organisation, including unions and nurses in wards. It seemed that more
networking had occurred as a result of a number of the processes introduced, for
example:
The use of representative CCDM councils at DHB and ward level, where all levels of
staff together worked on aspects of the programme with the overall aim of
achieving a safe and efficient workplace for staff and patients.
The availability of the detailed and regularly updated CaaG screens throughout the
hospital, providing a level of transparency of progress of patients’ journeys through
the hospital and tacit approval of nurses at ward level to take proactive steps to
prevent blockages in the system and increase efficiencies.
The whole of staff engagement in the workload assessment at ward level and
sharing of findings, which seemed to be a bonding experience for staff.
Anecdotally (as conveyed to evaluators), the redeployment of staff from one ward
or service to another as part of a variance response management activity leading
to increased staff flexibility. Increased communication, reciprocation and sharing
knowledge of other ways of working were also identified as an unexpected by-
product of VRM.
It is acknowledged by the evaluators that the CCDM programme to date has been
nursing focused and has not included other disciplines, except for midwifery, in direct
application of the tools into practice (other than the work analysis in a few wards and
VRM tool development). Therefore, reference to a ‘whole of organisation’ approach
may seem to overstate the impact of CCDM, However, nurses do make up the bulk of a
hospital workforce, and logically improvements for nurses would also likely impact on
other disciplines as a result of increased efficiencies.
The evaluators do acknowledge the work being undertaken by the SSHW Unit currently
with other disciplines to develop CCDM tools that meet their needs, while integrating
with the current tools. There has been progress made in a midwifery specific adaptation
to the programme, and an upgrade to TrendCare® will allow full roll-out of this in the
near future. There are a number of Discipline and Service Specific Advisory Groups
currently providing advice to the SSHW Unit on adaptations to the tools to meet their
specific needs. The evaluators noted general enthusiasm from allied health and
medicine in particular about having access to CCDM tools modified for their use.
Final CCDM Evaluation Report (January 2015) 73
From an economic perspective, concern has been expressed in various forums that that
there is the potential for CCDM to increase the cost of staffing. What we have found is
that CCDM provides a comprehensive infrastructure for a hospital to start managing its
nursing dollars better than it has done in the past, however it is difficult to compare the
before and after financials. Pre CCDM budgets were not necessarily based on any
scientific way of establishing true workforce needs to meet patient demand, nor were
they benchmarked nationally, as they are with the FTE calculation. For wards which
have completed the workload analysis and FTE calculations, the post CCDM financials
are likely to provide a more accurate reflection of the actual workforce requirements. It
is at this point that the hospital and wards should then work on efficiencies, as they will
have up to date, reliable evidence of the impact/cost savings. The few findings to date
on the budgetary implications of CCDM seem to indicate that it is relatively cost neutral,
because the reduction in use of casual staff, roster re-engineering and increasing skill mix
all contribute to balancing the required increases in FTE
With regard to the cost of staff turnover which is attributed to stress and an unhealthy
work environment (SSHWU COI, 2006), a 2012 study which examined the cost of turnover
in New Zealand supports the Mix and Match methodology premise that if the ward is
staffed to recommended levels, it can be cost effective (North et al, 2012). The study
identified that wards which were under budgeted FTE had a higher staff turnover, as well
as higher sick leave. The authors argued that for every two nurses who turnover, one
additional nurse could have been employed which may have prevented the turnover in
the first instance. Twigg et al (2013) concluded in recent research that the staffing
method of Nursing Hours Per Patient Day (NHPPD) is a cost effective initiative, as the
investment of increased nursing hours via the NHPPD staffing method had clinical
benefits and cost savings for improved nursing sensitive outcomes (NSO).
Concurrently with this evaluation, the Ministry of Health undertook a more detailed
financial analysis of the impact of CCDM. It was therefore decided that this evaluation
team would not undertake a similar analysis. The MoH indicated that they reached a
similar overall conclusion, that CCDM was relatively cost neutral.
Final CCDM Evaluation Report (January 2015) 74
Conclusion
The CCDM Programme provides a comprehensive infrastructure for a whole of hospital
approach to managing its nursing and midwifery workforce to better meet the needs of
patients. It enables critical analysis of historical hospital staffing resource allocation, fully
supported by both the DHB executive team and unions. The programme uses an
internationally validated electronic patient acuity tool to assess the pattern of staff
required to meet patient demand in each specific ward 24/7. A workload analysis tool
and FTE calculation tool inform roster re-engineering, including skill mix changes and
another suite of Variance Response Management tools provides ongoing review of
patient demand and an agreed response to unexpected demand.
In all, the programme incorporates a suite of 11 tools. The DHBs that choose to
implement the programme are supported at specific stages of implementation by
consultants attached to the SSHW Unit. In its current form, implementation of the
programme throughout a whole hospital is likely to take three to four years.
Programme roll-out has been much slower than predicted, initially owing to its
developmental nature. Processes used to engage the organisation as a whole were
necessary but time consuming, in many cases owing to the fact that the DHBs were not
as prepared as they initially thought. These initial processes, including mapping of all
other current DHB projects, assessment of staff readiness, a TrendCare® audit and the
allocation of resources for the programme, provided DHBs with a unique view of their
organisational preparedness to undertake a system wide approach to workforce analysis
and planning.
This evaluation has identified the various components of the CCDM programme and
provided feedback on their potential. The programme as a whole enables:
An open and transparent view via CaaG screens throughout the hospital,
providing a real time measure of the adequacy of nursing workforce capacity to
meet patient demand at ward, service and hospital level (the nurses can see
what’s coming and make contingency plans)
Agreed mechanisms to respond to variance in demand at the time by workforce
reallocation/redeployment (no questions asked activated between wards, usually
at early signs of variance)
A validated set of tools to re-engineer the workforce skill mix, roster and model of
care to better meet the needs of the clients, and measure the ‘fit’ in an ongoing
way
A suite of tools that enables a ward/ service/ hospital to respond to changing care
requirements and measure the effectiveness in an ongoing way
A better and more transparent base for budgeting.
It is the view of the evaluators that once a hospital has this programme in place and
keeps it tightly monitored (via an operations centre), they can then start to re-engineer
their patient flows, service delivery models, etc. with a better, real time view of the
impact on the workforce (and cost). A vital element of this whole programme that is
currently under-developed relates to agreement on the Core Data Set. This should be in
place to benchmark and monitor the work environment and patient outcomes, and to
provide evidence for change and the impact this change has on the organisation as a
whole.
At this point it is also worth noting the potential of this programme, if implemented
nationally, to provide a rich source of inter-DHB benchmarking and networking to learn
more about service improvement activities that do demonstrate effectiveness. Use of a
standardised electronic validated acuity tool that has been “New Zealandised” and
national agreement on the Core Data Set indicators would provide a unique opportunity
for monitoring and measuring national approaches towards achieving healthy
workplaces for staff and safe hospitals for patients, particularly in view of the double
challenge of an aging workforce and an aging population with more complex health
needs.
Final CCDM Evaluation Report (January 2015) 75
Recommendations
Following this 12 month evaluation, the evaluators recommend that the SSHW
Governance Group achieve a national commitment to rolling out the CCDM
Programme to all wards in all hospitals in New Zealand. The following recommendations
are made by the evaluators to modify and expedite the current processes.
1. Continue the CCDM programme
This programme provides a safe level playing field for front line hospital staff in the drive
to provide efficient and effective health services. The programme integrates well with
other quality initiatives. Fully implemented, it will enable national goal setting and
benchmarking.
DHB Chief Executives
1.1 All DHBs should implement the CCDM Programme.
2. Maximise and formalise the use of the SSHW Unit.
The SSHW Unit has a unique national overview of the functioning and potential of DHB
hospitals throughout the country. In rolling out the CCDM programme for DHBs, it
performs a vital function as a change agent. In order to maintain consistency, retain
highly skilled consultants and achieve efficiencies in programme roll-out, ongoing
development and benchmarking, the Unit needs to be retained on a permanent basis.
DHB Chief Executives
2.1 The SSHW Unit should become a permanent structure facilitating the programme
roll-out. Maximising the use of the expertise in the Unit will act to benefit the roll
out and further develop the programme in other service areas and disciplines.
2.2 The Unit should also facilitate national benchmarking activities and national
networking to support the change processes required.
2.3 The Unit should be resourced appropriately to undertake this role and achieve a
balance between development-focused work and support for current roll-outs.
Ideally a set of key performance indicators relating to the roll-out should be
developed for the SSHW Unit to report against.
SSHW Unit Director
2.4 Currently the SSHW Unit has a wealth of knowledge and experience in all facets
of the CCDM programme, with each consultant allocated a specific DHB.
Consideration should be given to the consultants specialising in components of
the programme and working collaboratively as an implementation team with all
DHBs.
2.5 It is recommended that the SSHW Unit, with its programme expertise, provide
centralised support and management of the workload analysis and FTE
calculation (Mix and Match Part 1 and Part 2) including analytical capacity, to
ensure a quick turnaround of reports.
Ministry of Health and DHB Chief Executives
2.6 Manage the negotiations of a national licence with the current validated patient
acuity tool provider, formally overview the management of the tool’s d
developments (to prevent hybridisation and different versions being in use
throughout the country), and facilitate access to upgrades.
Final CCDM Evaluation Report (January 2015) 76
3. Enhance the CCDM tools and processes.
At this point, the CCDM tools and processes should be viewed as a complete
programme. As such, the focus now needs to go on refining the tools and ordering their
implementation, so as to achieve the most effective and efficient implementation and
ongoing maintenance. The power point presentations, reports and associated
documents currently present the programme in an exceptionally complex way, and
need to be simplified.
SSHW Unit Director
3.1 Streamline the CCDM initial resource for DHBs, including an outline of their pre
and post CCDM resource requirements, particularly HR and IT resources, as well
as realistic timeframes.
3.2 Simplify the terminology and presentation of the programme, including the
reports. For example, consistently change Mix and Match Part 1 to Workload
Analysis and Mix and Match Part 2 to FTE Calculation.
3.3 Standardise as many processes as possible, including the provision of templates
to guide governance and planning processes, including report turnaround times.
3.4 Reconsider the order of the implementation process. For example, the Mix and
Match Part 2 FTE calculation could be completed in all wards prior to the
workload analysis, which may be considered only as a diagnostic tool for a
specific ward or service if necessary.
3.5 Adapt some CCDM information, assessment and training activities to be used by
clinicians in an e-learning environment, such as the Churchill Exercise. This would
allow staff to participate at a later date, for example when orientating.
DHB Chief Executives, Ministry of Health and Unit Director
3.6 Develop agreement on the Core Data Set nationally, and incorporate processes
to obtain reliable and regular reporting on these indicators early in the CCDM
implementation process. This would provide the DHB with a reliable set of data
against which they could measure the impact and benefits of the programme as
it rolls out, including staff satisfaction.
DHB Chief Executives
3.7 Support and encourage the hospital-wide use of the Capacity at a Glance
(CaaG) screen. Its widespread availability in public places for staff (and patients)
to view at their convenience was identified as the public face of the CCDM
programme.
3.8 Standardise the variance response management tools. It seemed that a
significant amount of time was spent customising these, although this made very
little difference in the end. Some DHBs were seeking permission to share.
4. Focus on completing the current roll-out in hospital wards in participating DHBs.
There is a risk that the SSHW Unit staffing resource will become dissipated as the DHB
programme roll-outs increase. Also interest in the programme has been generated by
other disciplines and services exposed to the potential of CCDM for them (for example,
allied health, mental health and midterm forecasting), requiring additional involvement
of the Unit.
The evaluation indicates that priority needs to go towards perfecting the system for
nursing and using the Core Data Set indicators to provide more conclusive evidence of
the direct impact of this programme towards achievement of the ‘triple aim’ in health
care. A full roll-out for nursing and midwifery (once the TrendCare® upgrade has been
completed) is likely to then enable fast tracking of adaptation and roll-out for other
disciplines.
Final CCDM Evaluation Report (January 2015) 77
Completing the roll-out in currently participating DHBs should take priority. Useful
learnings and efficiencies are likely to be gained for other areas once CCDM has been
rolled out to all wards in currently participating DHBs.
SSHW Governance Group
4.1 Dedicate priority resource to completion of full CCDM roll-out for nurse and
midwife staffing in all currently involved DHBs.
SSHW Governance Group and SSHW Unit Director
4.2 Negotiate agreed deadlines for continued implementation with currently
involved DHBs.
4.3 Work with DHBs that have agreed to implementation to ensure that their
executive team and middle management maintain their support and
involvement in the programme.
DHB Chief Executives
4.4 Continue the internal resourcing of the CCDM programme during roll-out until it is
embedded within the organisation as business as usual.
5. Develop support processes for those implementing change.
One very clear barrier to CCDM implementation, maintenance and roll-out is the level of
comfort staff have with change management. The CCDM programme at ward level
generally requires a change of service delivery model, roster re-engineering and the
introduction of skill mix. Calculating the impact of these changes and planning and
implementing them effectively require nurse managers to have a significant level of
leadership and management skills.
DHB Chief Executives, Unions and DHBs
5.1 Provide change management training for staff prior to CCDM implementation.
5.2 Establish and foster support networks between those embarking on changes and
those which have successfully completed changes. For example, facilitate
networks with nurse managers in similar settings who are undertaking changes,
following workload analysis and FTE calculation.
Final comments
The intent of this report is to present the CCDM programme as it is currently functioning
within DHBs. The programme provides a standardised and validated process for
matching and responding to the fluctuating, and at times, unanticipated demand for
patient care with the required workforce 24/7. If the ward/hospital/DHB does not
continue to maintain the programme, monitor its performance and respond
appropriately to the patient care demand on the day and over time, it runs the risk of
being viewed by nurses with scepticism. The programme will be blamed for ‘not working’
rather than the organisation(s) for not responding appropriately to an obvious staffing
deficit or surplus.
Final CCDM Evaluation Report (January 2015) 78
Appendix 1. Qualitative Feedback on the Programme
At each site visit the evaluators interviewed the executive team, attended a CCDM
and/or ward council meeting, interviewed the CCDM co-ordinator, toured the hospital,
including wards and the integrated operations centre, or met with duty management.
Focus groups were facilitated with charge nurses, union delegates and ward staff. In
preparation, each DHB received a standard set of questions based on the Programme
Logic framework, which were used to guide discussion and feedback with each set of
informants.
Feedback on Programme Progress
The key themes identified in Table 15 have been derived from an analysis of transcripts
from DHB site visits, focus groups and interviews. Summary themes are presented in the
table below.
Table 15. Summary of qualitative feedback gathered to benchmark CCDM progress
Qualitative
Feedback
Framework
National DHB executive &
management
Ward level Unions
Experience
and
impression
of the
CCDM
programme
Principles are sound.
Implementation is too
slow.
Language is confusing
and complicates the
programme.
Principles are sound
but most DHBs are
unhappy with level
of programme
development during
implementation
(although ideally
programme
implementation will
be with a
consolidated
programme).
Mixed feedback.
Positive in DHBs with
high organizational
and staff energy
and greater
organizational
spread. Negative in
DHBs with low
organisational and
staff energy and low
organisational
spread.
Expressed clearly
that the relationships
developed via the
partnership between
the unions and the
DHBs have been
priceless. This has
contributed to a
culture change; a
different way of
working based on
trust.
Feedback
on planned
impact of
CCDM
National DHB executive &
management
Ward level Union
Ultimate
intended
change
Vision for patient
safety directly
correlated to staffing
numbers.
Comprehensive
national programme
to achieve safe
staffing healthy
workplaces.
Vision varied at local
level; most common
themes were fiscal,
patient outcomes
and workforce
satisfaction.
The vision most
commonly
expressed was
individual job
satisfaction.
Staff are happy in
their place of work
and the workload is
reasonable and
safe.
Feedback
on
resources
National DHB executive &
management
Ward level Union
Financial DHBs contribute a
significant financial
resource to fund
SSHW.
Unions have invested
significant time and
financial resource.
Contributing twice,
once at national
level and with
resource required for
implementation.
DHBs identified that
they were often not
made fully aware of
the in-house cost (IT
and Business Analyst
resource and CCDM
coordination).
Minimal financial
impact at the ward
level.
Nurses unlikely to see
increase in FTE unless
budget allows for
this.
Minimal financial
impact for the
unions.
Final CCDM Evaluation Report (January 2015) 79
Feedback on
resources
(Cont.)
National DHB executive &
management
Ward level Union
Material CCDM resources are
perceived to be
inadequate and
needing
standardization and
improvement in
format.
Terminology
identified to be
problematic.
SSHW Unit resources
viewed to be
inadequate and
need
standardization
Terminology
identified to be
problematic.
Requesting more
research based
evidence.
Local resources
were produced in
the absence of
support/advice.
Often unaware of
spectrum of
resources available.
Terminology is
problematic.
Unions were
distributing NZNO
‘care point’ material
in many instances
and this could be
confusing at times
for the workforce.
Human SSHW Unit is well
resourced.
Advisory and
governance groups
consume human
resource.
Variable with some
DHBs employing
more resource than
others. Some union
delegates are more
involved than others.
Implementation
seems to be more
effective with more
resource for co-
ordination roles.
Variable with some
wards utilizing ward
staff and champions
more than others.
Union delegates
committed their
time to attend
council meetings
and champion
CCDM in the DHB.
Feedback on
Activities
National DHB executive &
management
Ward level Union
Most
valuable
tools
A validated acuity
tool.
TrendCare®, HaaG,
VRM.
TrendCare®, HaaG,
VRM.
Mix and Match Part
1 and 2 were
valued.
Wordload
analysis and
FTE
calculation
(Mix and
Match Parts 1
and 2)
Can be a barrier.
Needs
improvement.
Requires more
resource than
available.
Requires more
resource than
anticipated and
lengthy process for
feedback and
action. Can get staff
buy-in or
disengagement
depending on the
pace of the process.
Unions felt that Mix &
Match Part 1
engaged the
workforce but was
long and tiring.
There is a risk that if
Part 2
recommendations
to increase FTE are
not implemented
that the relationship
& partnership could
deteriorate & return
to a pre-CCDM
relationship with
each party having
their own agenda.
VRM (7 tools) Reported to be an
important
component of the
CCDM programme.
Very positive
response and assists
with culture change
and hospital
operations and
management
decisions.
Improves the skill of
the workforce with
redeployment,
although staff are
often initially
resistant. VRM assists
with culture change
as nurses & midwives
are not always
confident there will
be a response when
workloads exceed
capacity but in
some DHBS response
is rapid & gives the
workforce
confidence.
Unions stated that
this tool created an
integrated
workforce & reduces
siloing of wards and
nurses in wards. VRM
was states to
improve
communication
between wards.
Again there is a risk
that if a ward goes
red & there is no
response that trust is
lost in the
programme & DHB
management.
Final CCDM Evaluation Report (January 2015) 80
Feedback on
Outputs
National DHB executive &
management
Ward level Union
Progress with
implementati
on
Reports that
implementation is
too slow and spread
is too slow.
Has this progress
been cost effective?
Lack of mandate
limits progress.
Overall slow with
some exemplars with
high organizational
energy.
Slow and lack of
understanding.
Slow and under
resourced by the
DHBs.
Data Huge untapped
potential.
Identified need for
standardized metrics
to benchmark.
Request for
standardized metrics
to benchmark.
Allows for
transparency.
Data is consistently
questioned.
Allows for
transparency
Lack of
understanding at
the ward level
Data is consistently
questioned
Unions are engaged
with the data and
the transparency
has been beneficial
and helped build
trust in the
partnership.
Roster re-
engineering
Few FTE changes
overall nationally.
Intent is there but
lack of action.
When FTE increases
have been
implemented it has
been positive but if
identifies staff
overcapacity the
data is questioned
by the workforce.
If roster
reengineering
occurs and Part 2
recommendations
are met then the
DHB is seen to be
committed to the
programme and the
partnership remains
strong. Risk if Part 2
recommendations
are not
implemented of
deterioration of the
partnership and loss
of confidence in the
programme.
Feedback on
Outcomes
National DHB executive &
management
Ward level Union
Culture Change Understood and
occurs at a high
level.
Unions understand
the potential for
culture change.
Culture change in
partnership with the
union.
Lack of mandate
reduces potential for
culture change.
Profound when
CCDM is
implemented as a
system approach
hospital wide.
Culture change has
occurred via
partnership work
with the unions.
Profound when
CCDM is
implemented as a
system approach
hospital wide and
the workforce feel
supported by
executive
management.
Staff are
communicating
more between
wards and less siloed
as they move
around and ‘help
out’.
Profound when
CCDM is
implemented as a
system approach
and the union feels
that the data is
trusted and the Mix
and Match reports
recommendations
are implemented.
Culture change has
increased effective
communication with
the DHB and
improved
relationship and
when working well is
priceless.
Workforce Safe staffing is the
main driver.
Safe staffing and
staff satisfaction is
one of the main
drivers between
employer and union.
Positive changes
seen in some wards,
however not
widespread.
Positive changes
seen but not
widespread.
Final CCDM Evaluation Report (January 2015) 81
Feedback on
Outcomes
(Cont.)
National DHB executive &
management
Ward level Union
Main risks Implementation
does not spread.
Disengages
workforce.
Discussion may
return to mandated
ratios.
FTE calculations may
be fiscally
unsustainable.
Program is
unsustainable.
Scepticism from
workforce.
Union delegates
vote for return to
mandated ratios
argument.
Union delegates
vote for return to
mandated ratios
argument.
Patient safety A main driver
nationally.
Key driver for
executive.
Often overlooked at
the ward level.
Important driver for
unions
Staff satisfaction A main driver
nationally.
Key driver for
executive
A main driver at the
ward level.
Main driver for
unions.
Feedback on
Actual
Impact
National DHB executive &
management
Ward level Union
Financial An economic
impact due to
better utilization of a
scarce resource
Positive and actual
savings in ‘hard
green dollar’ terms
in a few DHBs.
Unsure of actual
financial impact in
majority of the
areas.
Majority unaware of
financial savings at
this level.
Minimal costs or
expenditure for
unions.
CCDM is
embedded
Starting in some
places but lacking in
the majority
Still a challenge as
not business as usual
in the majority
Not fully
implemented at this
level yet.
Progressing towards
this with NZNO.
Workforce is
more resilient
More required to
demonstrate
tangible change in
this area
Changes starting to
become evident
Some areas of
reporting beginning
changes.
Reports of beginning
of changes.
DHBs and Unions
work together
Relationship in this
form has been one
of the key benefits of
the programme
Improved
partnership at this
level and
meaningful
engagement
process
Engagement is
beneficial.
The change resulting
from this partnership
has been labelled
priceless when
working well.
Final CCDM Evaluation Report (January 2015) 82
Feedback on CCDM Programme Activities and tools
Feedback on the CCDM tools and activities was sought from all key informants on each
of the DHB site visits. A summary of this feedback is shown in Table 16, which addresses
general feedback about the CCDM tools and potential for enhancement of these tools.
Table 16. Summary of key points from feedback on CCDM tools following site visits
CCDM Activity/Tool General feedback Potential for enhancement
Pre-discovery phase: Often the actual impact of CCDM
implementation is not fully understood
at this point – including resource
requirements. It needs to be viewed as
a programme rather than a discrete,
finite project.
Greater assistance from the SSHWU is
needed to understand and articulate
the full resource requirements that will
be needed to undertake CCDM.
Discovery Phase: Staff surveys have not been repeated
post-implementation – hence queries
are often raised around the
importance of it. It is often at this point
that DHBs realize that their TrendCare®
data is not accurate enough to use to
inform the CCDM programme.
A TrendCare® audit and the need for
good quality data should be
emphasised more in the pre-discovery
phase. This should include how long
the improvement in quality may take
and the resources required to make
the improvements.
CCDM council This is a pivotal group for the
implementation and governance of
CCDM in the DHB.
Leadership from the SSHWU is essential
for the initial functioning of this group
and clear, easy to use resources
outlining the programme in its entirety
are essential.
CCDM local councils Local councils are one of the least
implemented aspects of CCDM. They
seemed to be implemented more
effectively when linked with ‘Know
how we are doing’ groups.
Benefits could be gained from looking
at where commonalities already exist
and utilising existing structures so as to
not add ‘another council’ or working
group.
Choice of pilot
ward(s)
Choice of ward not always well
considered. Often surgical or medical
ward.
Important to ensure that wards chosen
have an appropriate level of support
from both executive and charge level
prior to commencement.
Mix and Match Part 1
(Workload analysis)
Time-consuming at both ward and
analysis level – can be a bottleneck for
implementation. Mix and Match Part 1
gains buy-in from staff but length of
turnaround often disengages staff.
Improvements to the tool have been
ground fed and 3 DHBs have worked
with their business analysts to improve
the burden of coding inputting large
amounts of data. There needs to be
urgent refinement of the tool so that it
is no longer a manual process and
standardization may help reduce the
turnaround time for each report to be
completed.
Mix and Match Part 2
(FTE Calculations)
Another bottleneck in regards to
analysis and feedback of report. Also
can raise issues when change is
required but there is a lack of fiscal
resource to enable these changes to
occur. Mix and Match Part 2 does not
result in changes to the FTE or model of
care if the TrendCare® data is
considered to be inaccurate. In wards,
which have changed their model of
care and made changes to FTE the
results have been significantly positive.
Conduct this analysis centrally
(SSHWU?) and have a faster
turnaround time for feedback so as not
to lose engagement with staff. Ensure
TrendCare® data is accurate prior to
completing Part 2.
Make the report readable and
concise.
Churchill exercise Very inspiring for some DHBs and not
relevant for others – differing
perceptions of the importance of this
as a tool. A large amount of staffing
resource is required to attend the
exercise and if staff relocates, then this
knowledge is lost.
Have a DVD of this exercise that could
be shown post-discovery phase rather
than recommending all DHBs
complete exercise. This would also
allow for new staff to participate.
Final CCDM Evaluation Report (January 2015) 83
CCDM Activity/Tool
(Cont.)
General feedback Potential for enhancement
Core data set These indicators are already collected
by some DHBs and it is therefore felt
that there is not a requirement to have
a separate core data set from what is
currently collected. The analysis of the
metrics and feedback of this data
allows for transparency within the DHB.
This is a critical component of the data
feedback loop back to staff, This
transparency of the data assists
workforce buy-in for CCDM.
Build on existing DHB data set and
expand as required rather than have
as separate to Business as usual
processes. Standardization of the core
data set would enable benchmarking
between DHBs. Improve the format
that data is presented to the
workforce.
Capacity at a glance Universal feedback indicates this is an
important tool along with the culture
and communication changes it can
enable. This tool also contributes to
data transparency.
Make very clear at the CCDM business
case phase the IT requirements
needed to build and maintain screens.
Also need to have screens in readily
accessible areas rather than just on
PCs as this transparency is what
enables culture change and helps with
the spread of CCDM.
Variance Indicator
Boards
.
Again feedback stressed the
importance of this screen and the
need to ensure it can be easily
understood and conveyed to staff.
It would be helpful for the workforce to
have a greater understanding of what
the screen means and the need for
data quality to ensure that what is
being displayed is as accurate as
possible.
Reallocation policy
(Smart 5s)
Can be useful at start of CCDM
process for agreement on gifting and
redeployment between wards and
services. However not always used
within all DHBs.
Not always necessary for every
ward/service environment especially if
CCDM is well embedded and
deployment accepted.
Essential care
Guidelines
Many DHBs have these in place.
However they are often not
implemented, as the workforce is
reluctant to care ration. A ward may
be in ‘red’ but they have not changed
how the workforce is practising on the
day. For example, a ward in red may
stop doing washes and may prioritise
medications.
Further education in this area needs to
occur.
Integrated operations
centre
Not existent in every DHB however very
powerful mechanism and system when
well enabled and established within a
DHB.
There are tangible benefits to having
an actual operations centre in
existence – for the purpose of CCDM
and wider DHB capacity and
emergency planning. Enables cross-
organisation communication and
assists with visibility of demand and
capacity between wards and services.
Standard Operating
Procedures
Helpful for ease of communicating
pressure for areas within hospital.
When VRM occurs staff feel safe
knowing that the organization is
responding to the increased demand
for care. When there is no response to
an orange or red area the staff lose
trust in the organization and the CCDM
programme.
Processes need to be in place before
displaying these colours as red should
trigger a response rather than just
signal that an area in red is under
stress. Therefore a workable and
responsive escalation process must be
in place before this tool is used.
Final CCDM Evaluation Report (January 2015) 84
References
Aiken, L., Clarke, S., and Sloane, D. (2002). Hospital staffing, organisation and quality of
care: Cross-national findings. Nursing Outlook, 50, 187-194.
Brennan, C.W. and Daly, B.J. (2009). Patient acuity: a concept analysis. Journal of
Advanced Nursing, 65(5), 1114-1126.
Duffield, C., Diers, D., O’Brien-Pallas, L., Aisbett, C., Roche., M., King., M. and Aisbett, K.
(2010). Nursing staffing, nursing workload, the work environment and patient outcomes.
Applied nursing research. Elsevier.
Health Services Research Centre. (2013) A quantitative evaluation of the Mix and Match
staffing methodology. Safe Staffing Healthy Workplaces Unit. Wellington.
Kalisch, B., Tschannen, D., and Hee Lee, K. (2011). Do staffing levels predict missed
nursing care?. International Journal for Quality in Health Care, 23 (3), 302-308.
Kane, R., Shamliyan, T., Mueller, C., Duval., S., and Wilt, T. (2007). The association of
registered nursing staffing levels and patient outcomes: Systematic review and meta-
analysis. Medical Care, 45(12), 1195-1204.
Lankshear, A., Sheldon, T., and Maynard, A. (2005). Nursing Staffing and Healthcare
Outcomes: A systematic review of the International Research evidence. Advances in
Nursing Science.
Lawless, J. (2014). Documenting the Contribution of the SSHW Unit to the Joint DHB?NZNO
Safe Staffing Healthy Workplaces Agenda. New Zealand.
Moore, D., Blick, G., Leggott, J., Bloodworth, K.(2013) Assessment of the implementation
of the Productive Ward and Productive Operating Theatre programmes in New Zealand
Morrow, E., Robert, G., Maben, J. and Griffiths, P. (2011) Improving healthcare quality at
scale and pace. Lessons from The Productive Ward: Releasing time to care™
programme. Full report. Coventry, GB, NHS Institute for Innovation and Improvement.
Plummer, V. (2005). The Australian mandatory staffing experience. In D. J. Mason, J. K.
Lewitt, & M. Chaffee, M. (Eds). Policy and Politics in Nursing and Health Care (5th edition)
(pp. 527-540).
Preyra, C. (2004). Coding response to a case-mix measurement system based on multiple
diagnoses. Health Serv Res, 39(4,Pt1), 1027-45.
Safe Staffing Healthy Workplaces Unit (2012). Transforming the environment of care: A
DHB case study. Safe Staffing Healthy Workplaces Unit. Wellington.
Safe Staffing Healthy Workplaces Unit (2012) CCDM Background paper. Safe Staffing
Healthy Workplaces Unit. Wellington.
Safe Staffing Healthy Workplaces Unit (2013). Quantitative evaluation of the Variance
Indicator Board (VIB). Safe Staffing Healthy Workplaces Unit. Wellington.
Safe Staffing Healthy Workplaces Unit (COI) (2006). Report of the Safe Staffing/ Healthy
Workplaces Committee of Inquiry. Safe Staffing Healthy Workplaces Unit. Wellington.
Scoville, R., Little, K. (2014). Comparing Lean and Quality Improvement. IHI White Paper.
Institute for Healthcare Improvement. Cambridge, Massachusetts.