1

ENERGY SECURITY BOARD SYSTEM SERVICES AND AHEAD

MARKETS

April 2020

ContentsExecutive Summary .................................................................................................... 31. Introduction ........................................................................................................ 5

1.1. Purpose of the paper ................................................................................................. 5

1.2. Relationship to other work......................................................................................... 6

2. Operational challenges ..................................................................................... 72.1. Multi-faceted needs of a power system .................................................................... 7

2.2. Challenges in ensuring a secure and reliable system .............................................. 8

2.3. Need for change ...................................................................................................... 11

3. Establishing new system services ................................................................. 133.1. Need for additional system services ....................................................................... 13

3.2. Key system services to be established ................................................................... 14

3.3. Considerations for the establishment of a new system service framework ............ 17

4. Framework for procurement and scheduling of system services ................ 184.1. Ensuring resource availability for dispatch .............................................................. 18

4.2. Scheduling energy and system services in dispatch .............................................. 19

5. Ahead mechanism – general design considerations .................................... 225.1. The need for an ahead mechanism ........................................................................ 22

5.2. Additional benefits of ahead markets ...................................................................... 23

5.3. Overview of NEM ahead market design ................................................................. 24

5.4. Critical reform considerations - commitment design and impact on hedging contract25

5.5. High-level design trade-offs .................................................................................... 27

6. Ahead mechanism design options ................................................................. 286.1. Overview of spectrum of options ............................................................................. 28

6.2. Unit Commitment for Security (UCS) process ........................................................ 29

6.3. Option 1 - UCS-only option ..................................................................................... 33

6.4. Option 2 - UCS plus Voluntary Forward Market...................................................... 35

6.5. Option 3 - System security ahead market ............................................................... 37

6.6. Option 4 – Compulsory ahead market .................................................................... 40

7. Conclusions and next steps ........................................................................... 42A Abbreviations and Technical Terms ............................................................... 44

2

Executive Summary

While the NEM market design has performed well in the past, the future is very different. The future resource mix will be marked by a greater number of individual resources (from large- scale to millions of distributed energy systems). These resources are more diverse in their capabilities and operational performance exposing different technical and commercial challenges. Technology improvements, however, provide opportunities to create new value from these resources, and new business models built upon this opportunity. This operational environment, however, greatly increases the level of complexity in efficiently allocating resources and creates the need for flexible and adaptable regulatory and market frameworks to enable these opportunities.

As part of the post-2025 market design project, the ESB is identifying the market and regulatory arrangements that are fit-for-purpose under these future conditions. There are a number of inter-related reform considerations.

The post-2025 project has a major workstream to ensure that the power system continues to have available the technical properties essential for power system security and reliability of supply, and that these are provided in an affordable way. As some of these essential technical properties were previously provided as by-products of energy production, they are not all explicitly valued. This workstream is designed to support efficient investment and provision of these capabilities at the operational timeframe (from existing and new sources), mitigating the risks of potential supply shortages or costly interventions.

There are two components to ensuring the system will have the right mix of resources in real- time, and at lowest overall costs to consumers:

• Establishing new frameworks to value all essential system services so that they will beavailable to the power system when needed.

• Incorporating a mechanism in the NEM’s pre-dispatch and dispatch process that providesvisibility and enables efficient co-optimisation of the diverse set of resources ahead oftime to ensure all necessary system services will be available, without costly and distortionary interventions.

Essential system services

This paper sets out a framework for how system services can be best procured depending on the nature of the service. This includes mechanisms such as contracting services (for example: by TNSPs or AEMO), pricing and dispatch through a spot market, and regulation (including regulated tariffs and mandation). It is important that the mechanisms provide ongoing investment in resources that provide system services, which will be considered further in the ESB’s program of work.

Based on current experience and ongoing analysis of the operational needs of the future power system, high priority system services are described that are crucial for managing the power system with the rapidly changing generation mix. These include:

• Operating reserves to ensure adequate flexible dispatchable reserves are available tomanage variations in the supply and demand over a number of dispatch periods.

• Additional services for frequency management, particularly synchronous inertia to resistfrequency changes that would be too fast for frequency control services and protectionschemes to operate.

• System strength to ensure the power system can maintain and control a stable voltagewaveform during normal operation and following a disturbance.

3

The needs of the power system will change throughout the transition and this means that the framework and regulatory arrangements need to be flexible so the set of system services can be updated, without lengthy regulatory delays, to keep up with changing needs and, importantly, to capture and encourage technology improvements.

Ahead mechanisms

Market processes will need to be enhanced to coordinate the scheduling of all resources sothat the full needs of the system can be provided in a co-optimised way that minimises the cost of delivery. A form of ahead mechanism is considered essential for improving the visibility and confidence in essential system services.

This paper sets out a number of reform options that increase the amount of firming andflexibility in the system. The options range from strengthening the commitment mechanism for reliability and security, to ones that are more extensive but integrate fully with the energy market. This allows co-optimisation of energy and system services, better facilitating two-sided markets and cross-market coordination (such as with gas) and optimising a much more complex and diverse resource mix, potentially leading to overall lower cost of electricity for consumers.

Over the coming months, the ESB and market bodies will further develop these designs, evaluating the various options to identify a recommended design by the end of 2020 to form part of the overall post-2025 market design.

Further development of the design options will include how the new mechanisms will integrate with participant and system operator processes, including forecasting, bidding, information requirements, real-time operations, and settlement. This will also include analysis of the appropriate timing of any new mechanism/s to account for the required commitment level of the resources to ensure the services will be available in real-time.

Assessment criteria will be developed building on the draft assessment framework published in the post-2025 Issues Paper, and the framework will be used to evaluate the design options. This will include an assessment of the potential costs, benefits and trade-offs of the various options.

Staged implementation and interim measures

The recommendation will include a practical, staged implementation to make improvements as early as possible, while considering transitionary needs (such as to allow the forward energy market to evolve to the new design) and recognising important interdependencies with other policy initiatives, such as locational pricing, investment mechanisms for reliability, innovation to benefit the consumer, integration of DER, and two-sided markets.

In the meantime, the ESB has identified some actions that, while they do not address all the issues and do not replace the need for permanent and enduring solutions, can improve visibility and commitment of resources, as well as improve the current processes to secure the system. These interim measures are outlined in a separate paper to the COAG Energy Council.

4

1. Introduction

1.1. Purpose of the paper

Like all global electricity markets experiencing rapid change in their resource mix, the National Electricity Market (NEM) is facing growing challenges to meet its system security and reliability needs, while at the same time keeping prices at an affordable level. A secure and reliable system requires all elements of the power system to be operating within the technical envelope and to always have supply balancing demand. Affordable electricity requires appropriate investment and dispatch mechanisms to procure and schedule the efficient resource mix to meet all system needs and electricity demand at lowest cost.

In the face of the challenges of managing the transition of the power system, the COAG Energy Council requested the Energy Security Board (ESB) to advise on a long-term, fit-for-purpose market framework to support reliability. The NEM should be modified as necessary to meet the needs of future diverse sources of non-dispatchable generation and flexible resources including demand side response, storage and distributed energy resource participation.

In December 2019, the COAG Energy Council also asked the ESB to provide advice on prioritising initiatives currently underway that are aimed at identifying the range of essential system security services required as the NEM transitions, how these services can be valued, and how they can be efficiently procured. This should also include advice on arrangements to give visibility of available resources and options to ensure their sufficiency, including ahead markets and unit commitment runs, to ensure that the right resources are available at the right time and to ensure co-optimisation of necessary resources and services. Advice on preferred frameworks is to be provided by March 2020, with detailed analysis by end of 2020.

This paper sets out a framework to ensure the system will have the right mix of resources to operate in a secure and reliable way, and at lowest cost possible to end consumers. There are two components:

• Establishing new frameworks to value system services and to ensure that they will beavailable to the power system when needed in recognition of the changing power systemneeds.

• Incorporating a mechanism in the NEM’s pre-dispatch and dispatch process that provides visibility and enables efficient co-optimisation of a higher complexity and more diverse setof resources ahead of time to ensure all necessary system services will be available, without costly interventions.

This paper describes the priority system services that need to be established for managing the power system with the transitioning generation technology. Importantly, however, is the framework itself and regulatory arrangements that will allow the system services to adapt, without lengthy regulatory delays to the changing needs of the power system and to capture technology innovation that can provide new ways of meeting those needs. This framework will be developed further in the coming months. The framework will consider both the efficient use of existing resources and that the procurement instruments or market signals can encourage new resources to be built with the right capabilities. The procurement of these services and the investment structures associated are an important element to ensuring their availability; this is being considered in a related piece of work, and will be incorporated into future considerations, with recognition they are also dependent on the structure and signals in the dispatch market.

The potential options for changes to the dispatch1 mechanism that incorporate an “ahead” element to have these services scheduled are then discussed. Each of these options introduces

1 Throughout this document, dispatch has a general meaning referring to the process of scheduling, committing, andproviding targets in the electricity market.

5

important benefits but also have trade-offs that will be worked through and evaluated in the coming months to reach a final recommendation by the end of 2020.

1.2. Relationship to other work

This work is being progressed in parallel with other pieces of work the ESB and the market bodies are also progressing, which are needed to understand the changing needs of the physical power system and regulatory reforms to meet these needs.

AEMO has been examining the evolution of the NEM through the Integrated System Plan (ISP) and technical challenges in the Renewable Integration Study (RIS). The RIS provides important context to the technical system limits that could present barriers to secure and reliable system operation with the expected penetration of wind and solar (both utility and rooftop) generation in the near-term; and insights into actions to overcome these barriers and enable the transition at lowest cost.

The system services and market frameworks described in this paper are built upon ongoing insights derived from the RIS analysis, aiming at creating market and regulatory processes to overcome the potential operational challenges through the efficient procurement of system services and scheduling of resources.

The AEMC is reviewing system security mechanisms and other policies through rule changes and reviews, such as ensuring that primary frequency control is available, reviewing current system strength arrangements, and has recently made a draft determination to not proceed with a short-term forward market for the NEM.

These pieces of work will fit within the broader post-2025 program of work, which amongst other considerations, is looking at mechanisms to ensure adequate investment, enabling the use of the demand side through the creation of two-sided markets and integration of distributed energy resources (DER), as well as the COGATI reforms that might introduce locational marginal pricing. Ahead markets and enhancements to co-optimised scheduling have the potential to broaden the participation of consumers in two-sided markets.

6

2. Operational challenges

2.1. Multi-faceted needs of a power system

The objective of managing the power system is to ensure secure and reliable2 electricity delivery to end consumers at lowest cost. This is a complex operation, and while much of the public discourse on energy markets focuses on having enough energy supply to meet demand, there are a multitude of system services required to keep a power system reliable, secure, and resilient as set out in Table 1. These include the availability of operating reserves, management of frequency and voltage, and system restoration.

Table 1 Power system requirements in the NEM

Service Power system requirement

Energy Provision of supply to meet demandOperating reserve Capability to respond to changesExisting FCASAdditional frequency services (e.g., faster frequency control & inertial response)System strength

Voltage control

Ability to maintain frequency stability

Ability to maintain voltage stability

System restart services Restart the system after interruption

The services are inter-related, in the sense that some resources are able to provide many of the listed services and it can lead to either:

• The provision of a service by a resource inherently leading to the provision of anotherservice (for example, a synchronous machine that provides system strength could alsoprovide inertia).

• The provision of a service reducing the need for another service. (For example, havingmore inertia in the system could potentially mean less fast frequency response is needed,as more inertia would reduce the rate of change of frequency.)

The way that the services are specified also depends on the needs of the power system at the time (which can change as the transition continues) and with engineering and technological innovations able to meet the needs in new ways.

Crucially, the availability of these services in some form is critical to keeping the power system stable and to respond quickly to changing system conditions without large-scale disruption. Ensuring adequate supply of all services is a key design challenge for the market to enable an orderly and cost-effective transition in the NEM’s generation mix.

2 A system is secure if it is operating within its operational envelope including voltage and frequency limits and hasenough margin so that it can be quickly returned within the limits following a credible contingency event. Reliability means having enough generation capacity, including those being dispatched and in reserve, to meet demand so that no involuntary load shedding will occur.

7

2.2. Challenges in ensuring a secure and reliable system

The changing generation mixThe NEM is experiencing rapid changes in its generation mix, with traditional synchronous resources (coal and gas) retiring and being displaced from dispatch by increased penetration of large scale variable renewable energy (VRE) sources (wind and solar) and ‘behind-the-meter’ or distributed energy resources (DER).3

As per the draft 2020 ISP, the expected closure of coal generators sees around 15 GW (63%) ofcoal-capacity retiring by 2040.4 In the same time period, wind and solar generation capacity inthe NEM could triple from 15 GW in 2018-19 to 45 GW in 2039-40, and distributed energyresources are expected to double or triple by 2040, meeting 13 to 22% of annual consumption.

Although the annual share of different technologies is commonly quoted and is an important metric to understand the changing market for system operation, the generation mix at any point in time is also a critical metric for operational decisions in the market. For this we use the “instantaneous” penetration of inverter-based generation. With an increase in wind and solar generation capacity, the instantaneous penetration of inverter-based generation is expected to rapidly increase. The RIS highlights that the maximum penetration in 2025 is forecast to regularly exceed 50% NEM-wide and could at times reach much higher levels in all regions. South Australia is already world-leading having reached instantaneous 142% penetration.5 Minimum operational demand is also reducing due to the penetration of DER and could reach critical levels if measures are not put in place.6, 7

The resource mix in the future will be very different to what it has been to date. Australia is already shifting from a small number of large centralised power sources to a much larger number of more geographically diverse resources and millions of distributed energy resources that are supported by sophisticated energy management systems. While coal-fired generation will progressively retire, the remaining coal units are expected to cycle more (intra-day, days at a time or across a season) along with baseload gas generators when inverter-based generation is high. Energy storage will have a large instantaneous capacity in aggregate but is limited in terms of its energy storage – particularly battery storage systems. Gas-powered generation, together with energy storage, will have a greater role in supporting the system, with the needs being very different in days (and seasons) with relatively high wind and solar output and times where the wind and solar availability is lower or more intermittent.

3 Conventional thermal generation is synchronous generation because of the way that they generate electricity bymachines that spin consistent with the system frequency and connect directly to the grid. On the other hand, VRE and batteries do not have these spinning machines but are connected to the system by power electronic devices known as inverters. This gives way to different electrical properties, with synchronous generation inherently providing system stability that inverter-based resources currently do not.

4 AEMO (December 2019), Draft 2020 Integrated System Plan, https://www.aemo.com.au/-/media/Files/Electricity/NEM/Planning_and_Forecasting/ISP/2019/Draft-2020-Integrated-System-Plan.pdf

5 AEMO (October 2019), Maintaining Power System Security with High Penetrations of Wind and Solar Generation:International insights for Australia, https://www.aemo.com.au/-/media/files/electricity/nem/security_and_reliability/future-energy-systems/2019/aemo-ris-international-review-oct-19.pdf

6 AEMO (March 2018), AEMO Observations: Operational and market challenges to reliability and security in theNEM, https://www.aemo.com.au/-/media/Files/Media_Centre/2018/AEMO-observations.pdf

7 AEMO (March 2019), Integrating Utility-scale Renewables and Distributed Energy Resources in the SWIS,https://aemo.com.au/-/media/files/electricity/wem/security_and_reliability/2019/integrating-utility-scale-renewables-and-der-in-the-swis.pdf

8

As a result of these fundamental shifts, it is critical that when assessing the ongoing needs of the power system and the market and regulatory arrangements, it is done against the future resource mix and advances in technology and business models.

Increasing variability and uncertainty in demand to be met by dispatchable sourcesAEMO’s RIS has identified significantly greater variability and uncertainty in the power system in the future. Variability is characterised as the magnitude of changes in the supply-demand balance over time (which is driven by both variability in the supply and in the demand profiles). Uncertainty is the inability to accurately predict future demand, supply, and grid conditions, which includes the demand for system services. With increasing influence from changing weather conditions on the power system, the ability to forecast the system is becoming increasingly challenging. For example, the exact timing and size of a change in wind conditions can be difficult to predict ahead of time but can drastically influence the required configuration of the power system to continue to operate reliably and securely.

Synchronous resources remain crucial for power system securityThe NEM, like all large-scale AC power systems, requires synchronous resources. The global electricity supply industry is researching and running trials to reduce reliance on carbon-intensive synchronous generation to decarbonise electricity systems, but there are currently no gigawatt scale power systems that do not rely on synchronous resources for grid stability and resilience. While new VRE renewable and battery technologies can provide some services and may in future provide more services, they cannot currently replace all the security services provided by synchronous resources, such as system strength and inertia. It may be that other types of technologies can provide these in future. While no specific service definition can be future-proof, the framework should be adaptable to change with technology innovation.

System impact of declining online synchronous capacityAs synchronous units come offline, their contribution to system security is removed from the system. This dynamic results in times where there could be insufficient essential technical capabilities despite abundant energy supply. For example

• Reductions in the number of synchronous generating units online reduces the amount ofsystem inertia.8 AEMO has observed a decline in system inertia and is expecting this todecline further as wind and solar penetration increases. Less inertia means system frequency changes faster for disturbances, and energy will need to be replaced faster. Operation at very low levels of inertia is uncharted territory for a large power system and is dependent on a number of inter-related factors, such as primary frequency response, load relief, and frequency control reserves. As such, the system could be at risk unless appropriate system operation and market design ensures it has enough frequency management resources to operate at lower inertia levels, maintain the stability of the NEM, and increase system resilience to unforeseen events.

• System strength is also provided by synchronous machines (generators or condensers)and helps maintain stability of the system within the area they are connected, both after adisturbance, as well as under normal operation and switching of network elements. System strength provision is relatively locational. Therefore, adequate supply must be distributed throughout the NEM for its secure operation. AEMO is already intervening on a frequent basis in parts of the NEM to ensure resources are online to maintain adequate levels of

8 Roughly speaking, synchronous generating units have a massive rotor that spins at grid frequency and helps resist frequency changes (synchronous inertia). This is distinct from asynchronous generation sources that use electronic converters to connect to the system.

9

system strength, when the energy price is insufficient to economically dispatch these resources under the current dispatch mechanism.

• As synchronous generators often operate with some headroom and can change theiroutput when needed, they are a significant source of operating reserve. VRE can also veryquickly change its output but it is normally operated at the maximum available capacity at any time (subject to network and other system constraints). Therefore, VRE do not normally supply operating reserve. Given VRE generators’ maximum capacity depends on the changing weather conditions, the increasing uptake of VRE and distributed resources increase the demand for operating reserves because of the higher variability and uncertainty in the system at all timescales.

Challenges in procuring adequate resources in dispatchThe current NEM market design is being challenged to efficiently meet the evolving needs of the system at an operational timeframe due to:

• A lack of explicit procurement and scheduling of many system services (other than energyand FCAS) within dispatch.

• The reliance on a real-time only market design to incentivise resource provision andcommitment to meet system needs.

The NEM's dispatch process was developed to find the least cost solution to dispatch resources to meet the load (energy) and later evolved to add a real-time frequency response (FCAS) procurement to manage the impact of credible contingency events co-optimised with the dispatch for energy.9 The dispatch process does not currently value or enable the scheduling of the full range of system services that are essential for the secure operation of the grid.

The lack of holistic scheduling by the current dispatch process has been demonstrated by the ongoing system operator directions occurring in South Australia. At times of high VRE generation, the energy price in South Australia reduce due to the VRE’s near-zero short-run marginal cost. At these times, the energy price is signalling plants with higher operating costs to reduce their output, displacing synchronous generating units from the merit order. What the energy price does not recognise, however, is that these units are required to stay online to provide system strength. The only mechanism available to compensate for their operation is for the system operator to use the directions process, which is an intervention.

The amount of flexible operating reserves is also not managed and co-optimised explicitly within the dispatch process. Instead, AEMO assesses the adequacy of future reserves using the pre- dispatch and STPASA10 information and, in times of forecast reserve shortages, declares a lack of reserve 2 (LOR2) condition11. This notice asks for a voluntary market response that is not explicitly remunerated. If the LOR2 declaration is not met by sufficient market response, AEMO intervenes through contracting and dispatching RERT or directing generation. While participants are more likely to make reserves available in anticipation of high demand or energy prices, a market response to LOR2 is less likely when reserve shortage happens during low demand but high VRE conditions and when pre-dispatch energy prices are low.

Other services (such as voltage control) can be procured (and hence remunerated) though the network support and control ancillary services (NSCAS) framework. NSCAS is an instrument that

9 The dispatch is security constrained and respects the physical limitation of the transmission system.10 STPASA stands for Short Term Projected Assessment of System Adequacy and is produced by AEMO on a 7-day

rolling basis based on information provided by generators as to their maximum availability.11 In a lack of reserve 2 (LOR2) condition, the amount of reserve in the system is or is forecast to be less than the

greater of largest contingency and forecast uncertainty measure, meaning the system is not secure against credible contingencies.

10

can be used to contract with existing resources and has the advantage that it is a flexible framework and allows for bespoke solutions to potential system security gaps.

New provisions exist for Transmission Network Service Providers (TNSP) to address (in their planning processes) forecast gaps in minimum levels of system strength and inertia, when these are identified by AEMO. There are currently gaps identified in three of the NEM’s five regions.

The current NSCAS, as well as the system strength and inertia frameworks, do not provide a clear and transparent value to the provision of system services when gaps have not been identified. As such, an investor in new resources would not be able to incorporate potential returns from service provision unless the investment decision was being made at the time a gap is in force. For example, an investor looking to build a new gas turbine could include the capability to operate in synchronous condenser mode and thereby offer system strength and inertia services. However, this incremental investment would only be valued if a gap had been established and a RIT-T or NSCAS tender was active.

Even when resources are contracted, and a reward mechanism exists, there is no market process to commit and schedule units in dispatch to ensure that minimum levels are met in the operational timeframe and the delivery of multiple services is co-optimised. Instead, the system operator manages the dispatch of contracted services via a manual process using non-committedpre-dispatch information.

The current NEM is based on a real-time only and self-commitment design in which participants can change their synchronisation and de-synchronisation decisions at very short notice. With the declining synchronous capacity and increasing complexity of scheduling multiple services in the future, this is more likely to cause a shortfall in some system services with very short notice to the system operator. Given many synchronous units have significant start-up times, this could leave little time and reduced options to intervene, leading to increased risk to the system and higher costs. Further, as the system relies more heavily on storage systems to balance the system, these will also need to be in the appropriate state of charge to provide the right service in real- time.

2.3. Need for change

As the current and future NEM has a very different market dynamic compared to when it was initially designed more than twenty years ago, there is an urgent need to reconsider the various design aspects to ensure that reliability and security of the system can still be efficiently met as the market further transitions its generation mix.

As discussed above, co-ordination of resources that provide system services is crucial during this transition and becoming more complex. It is no longer adequate for the market to only be based around co-ordinating the supply of energy and FCAS.

If participants make their operational decisions on when to run their plant based on expected energy and FCAS market revenue alone, they will want to go offline when market prices are expected to be low. The market actively incentivises plant with higher energy costs to decommit at times of high VRE output where the short-run marginal price for provision of energy is near- zero. While this is the right signal for energy, it does not reflect the true value that resources provide in supporting system security and reliability.

The dwindling online synchronous resources, combined with the lack of scheduling and commitment for critical system services in dispatch, will lead to a greater reliance on intervention to ensure the right mix of resources are available before real-time. The unpredictable nature of such intervention represents further distortion to the market and could cause further disruption to participants’ operational planning and therefore potential costs to consumers.

The electricity system needs to remain secure and reliable while delivering electricity to end consumers at lowest possible prices during the transition. The introduction of additional, or the

11

enhancement to existing market and regulatory mechanisms (e.g., regulating a framework for network or system support contracts) will facilitate the procurement of the system services crucial to the secure and reliability operation of the system. Enhancement to the dispatch mechanism through appropriate ahead scheduling and commitment processes will more efficiently co- ordinate the provision of system services with energy in real-time, which will lower the cost of electricity.

12

3. Establishing new system services

3.1. Need for additional system services

The current NEM design co-optimises the dispatch of energy and FCAS services for each dispatch interval in real-time. Participants provide offers for energy and FCAS services, which are used by the NEM Dispatch Engine (NEMDE) to find the most economically efficient dispatch. The dispatch is constrained by the physics of the power system, including the constraints on individual resources, components of the network, and the characteristics of the system as a whole(security-constrained economic dispatch).

All other essential services required to maintain security and reliability are either:

• Provided as a free by-product of the resource being online and generating energy.

• Provided via regulatory requirements (for example, technical standards, includingconnection requirements12 and those for building and maintaining assets).

• Contracted by TNSPs or AEMO in network support and control ancillary service (NSCAS)contracts or other non-market contracts.

• Procured by out-of-market intervention by AEMO such as RERT and market directions. As discussed in Section 2 and explored in more detail in AEMO’s RIS, the change in thegeneration mix is making some services that used to be abundant, now scarce at times. In a well- functioning electricity market, all services critical to the reliable and secure supply of electricity should be available whenever needed in real-time.

Currently energy and FCAS are the only services that are both valued and remunerated and explicitly co-optimised in dispatch. Some services outlined in Table 1 are either not explicitly remunerated at all (e.g., operating reserve), or are remunerated through a contract with AEMO or via the TNSPs (e.g., system strength and inertia). The current valuation framework for system strength and inertia, however, places a positive value on the service only to the extent that it fills the identified minimum level gap and only during the time that the gap is being filled by the TNSP. It does not explicitly recognise the incremental cost of the ongoing resource transition, which moves the system closer to the minimum level gap and hence the next round of capital investment.13 It also does not recognise potential value of new generation investment including capabilities to more efficiently reduce the cost of provision of system services (e.g. by including the ability to operate in synchronous condenser mode when not generating).

The AEMC has initiated a review of the current system strength framework, which will investigate potential measures to improve on the current process.14 Further, none of the services in Table 1 are explicitly integrated into the dispatch process. That is, NEMDE does not directly schedule or commit resources to provide the services for the current or future dispatch intervals. At the dispatch timeframe, the provision of these services either relies on AEMO’s manual intervention (e.g., direction for system strength or RERT for reserves) or activation of NSCAS contracts (e.g., for provision of voltage control). While the dispatch process allows the system operator to bring

12 New generators wishing to connect to the NEM must prove they are capable of meeting the generator performancestandards and complying the technical requirements as set out by AEMO procedures and TNSPs. This includes, amongst other things, a requirement to “do no harm” to the fault levels of the surrounding area.

13 For example, an alternative approach could use the Turvey (or perturbation) LRMC method, which is more likely togenerate a stable value for such services.

14 AEMC (April 2019), Investigation into system strength frameworks in the NEM,https://www.aemc.gov.au/market-reviews-advice/investigation-system-strength-frameworks-nem

13

on these services to address gaps in operational timeframe, the lack of co-optimisation with the rest energy and FCAS market means that these services are unlikely to be provided at least-cost.

Potential providers might not adequately supply a service if it is not remunerated or if it is not explicitly scheduled in dispatch. This will likely lead to an increasing frequency of AEMO interventions, causing further distortion and inefficiencies in the long term. From the perspective of the market and regulatory frameworks, this translates into the need of a mechanism to specifically value, procure, and remunerate these system services (beyond the current energy and FCAS markets and NSCAS-style contracting), and a mechanism to dispatch them.

While all services presented in Table 1 are needed if the system requirements are to be met at all times, there is an urgent need to establish formal market mechanisms for key system services and incorporate them into scheduling. These services are likely to be needed regularly given the expected future market outlook. Therefore, it is crucial that they will always be available in real- time to keep the system reliable and secure without regular intervention from the system operator. Procuring and scheduling them through a formal mechanism will have the following benefits:

• Ensuring the services can be explicitly scheduled within dispatch, committing the mostefficient set of resources that can meet the needs of the system at the time required. Thiswill also avoid the need for AEMO to make ad hoc out-of-market directions for the provisionof these services, reducing dispatch risk and allowing a market response to reduce overallcost.

• Allowing providers of the services to be explicitly remunerated, such that the cost ofproviding the service can be recovered. This will allow these providers to remain in businessfor as long as it is efficient to do so. It also creates opportunities for new technologies tovalue stack, by creating additional revenue streams and encouraging new technology to meet these physical needs.

• Generating transparent prices, providing signals for investment in technology with thecapability to provide the services.

The rest of this section will provide a high-level overview of the key system services that have been identified as highest priority at this time, with more detailed discussion on the procurement and scheduling framework to be found in Section 4. The proposals here are based on current understanding noting that relevant engineering studies are ongoing. Further engineering and market studies are needed to guide detailed product design. As understanding progresses, there may be a need to add new services or change the way that services are considered. As such, the framework needs to be flexible and adaptable to changing system conditions and new technologies and developments in power system engineering.

3.2. Key system services to be established

Operating reservesOperating reserve is a product proposed to ensure there is adequate spare dispatchable capacity in the system to provide flexibility in real-time. The RIS has shown that variability in the study year of 2025 is expected to be significantly higher than now and, combined with the inherent uncertainties in forecasts, there is a need for additional reserve in the system in order to respond to unexpected changes in supply and demand over a range of timescales.

Dispatchable reserve capacity available to central dispatch provides headroom to manage variations in supply and demand as well as to give more options to recover from power system events. This is similar to the creation of another FCAS product to manage changes over a longer time period, but it is different in that operating reserve does not have to be frequency responsive. It needs to be ‘firm’ and dispatchable and, may be specified to have certain ramp rate characteristics.

14

Operating reserve is a well-known product internationally – but it has different meanings and designs in different markets. In the US operating reserve is used to provide contingency FCAS services, as well as dispatchable capacity in the system, with response times of generally 10 or 30 minutes. Additional reserve products, such as flexible ramping products and imbalance services, have also been introduced in some markets (such as California) to adapt to the increased variability and uncertainty by ensuring sufficient dispatchable reserves are committed to provide the flexibility in real-time.

The NEM differs from many other markets globally in that it relies on participants managing their own portfolios to provide additional reserve beyond the current dispatch interval, without explicitly remunerating such provision. Individual participants typically leave spare headroom in their portfolio to protect themselves against unexpected changes (such as a plant outage) that would leave them short against their contracted position. This decentralised and voluntary approach assumes that all participants collectively have the right incentive to provide the efficient reserve level with the appropriate flexibility to manage system changes. In a system with an abundance of dispatchable capacity and with uncertainty mainly due to load forecast error and contingencies, this approach has been acceptable without significant system operator intervention. However, with increasing variability and uncertainty in supply and demand and with pressure on synchronous capacity that provides the headroom, this approach may not be sustainable in the future. Without enough operating reserve, the system might not be secure or reliable when the supply-demand balance changes unexpectedly. With increasing variability and uncertainty, there could be more instances where there is a forecast of insufficient reserve to protect against the largest contingency, forecast uncertainty or the need to increase supply over multiple dispatch intervals (ramping need), where AEMO will issue a lack of reserve notice. In the absence of explicitly remunerating participants for additional reserve provision, there might not be an adequate market response, forcing AEMO to intervene through RERT or direction in the market, which could be costly to consumers.

The introduction of explicit operating reserve services addresses this. In the event of an unexpected reduction in supply (or increase in demand), firm spare capacity, which can be sourced from generation, storage or demand response resources, would be available to be dispatched to keep the supply demand in balance and ensure the system is secure. Different services could be defined for different time frames beyond a dispatch interval (e.g. 10, 30, and 60 minutes) and can take into account ramping requirements. In this regard, operating reserve constitutes a ‘firming’ product. The product design should also ensure the amount of reserves procured and the value placed on them are consistent with market conditions, so that additional reserve is paid for only when needed and at the price consistent with its value.

Cost recovery can also be designed to place the obligation with the parties that most contribute to the need for operating reserves. For example, the extent to which participants are exposed to the costs of operating reserves based on their individual variability will provide an incentive to reduce the variability and uncertainty through better forecasting, bidding practices or even installing equipment, such as batteries, to manage their variability. An appropriately designed cost recovery mechanism for operating reserves would introduce incentives to reduce the need for the product where efficient.

Frequency managementNEM dispatch co-optimises the procurement of FCAS with energy production every five minutes to assist with frequency management. Outside of these market mechanisms, the system also inherently relies on primary frequency response (PFR) and on inertia provided as a by-product of energy production.

To address a reduction in system inertia from a declining synchronous generation fleet, a framework has been put in place whereby if AEMO identifies a potential shortfall in a planning horizon, the TNSP in the region is responsible for filling the gap. To address a reduction in PFR, the AEMC has made a draft rule to require all scheduled and semi-scheduled generators in the

15

NEM to support the secure operation of the power system by responding automatically to changes in power system frequency.15 AEMO and the AEMC have also committed to a joint work program initiated by the AEMC’s Frequency Control Frameworks Review (FCFR) to better understand the services provided by the current FCAS suite and as by-products of energy production.

AEMO is working to understand the technical requirements for additional frequency management products to maintain a secure power system. In particular, the capability of the transitioning generation fleet needs to be further examined in all operating system conditions, to understand the response of the inverter-based technologies to power system events and their technical capability to provide the additional supporting services.

Technical studies will allow for the specific products to be defined and to understand the trade- offs between various products. For example, the following two potential services could be introduced, aimed at ensuring that system frequency does not change too quickly such that protection systems can still operate as expected:

• Synchronous inertia – which pays for resources that provide synchronous inertia to be on-line to resist frequency changes. This can be provided by synchronous generators andcondensers. Note that complementary services, such as FCAS, are required to arrest and restore frequency.

• Fast frequency response – which pays for resources that can change their output to arrestfrequency change but respond faster than the current 6-second FCAS services. This couldbe supplied by a wide range of resources including VRE generators and batteries.

In introducing these products, one needs to carefully consider their interdependencies. For example, as more inertia means that the frequency deteriorates less rapidly following a contingency, a fast frequency response product might not be needed if there is sufficient inertia provided by synchronous resources. If fast frequency response products are needed, it could involve the creation of a new product or the redefinition of the 6-second FCAS to a faster response (e.g. 1 second).

System StrengthSystem strength is perhaps the most notable security service that is declining with less synchronous resources online. Given the NEM’s long and stringy network topology, the system is experiencing low system strength ahead of international power systems. Low system strength can lead to a variety of system issues, including failure of protection systems and instability.

As system strength can currently be practically supplied only by synchronous generators or condensers at specific locations, and the specific array of technical requirements is complex and dependent on the circumstances of the system, it may not be possible to rely on the current dispatch process to signal the marginal value for this service. To address this concern, new frameworks were introduced to the NEM in 2017 including:

• The minimum system strength framework – which obligates TNSPs to procure systemstrength services to fill a shortfall if this has been declared by AEMO.

• The ‘do no harm’ framework – which requires TNSPs and generators to assess adverseimpacts of a new generator connection on system strength and for the generator to putremediation solutions in place.

However, even with sufficient investment to ensure the capability exists, it may still be possible to optimise the scheduling of the service. The provision of system strength currently can also lead to

15AEMC (December 2019), Mandatory primary frequency response, https://www.aemc.gov.au/rule-changes/mandatory-primary-frequency-response

16

a provision of some inertia. If system strength is provided by a synchronous generator, the same resource could also produce energy, FCAS, and operating reserve. Therefore, there may be an opportunity to co-optimise the dispatch of these services such that an efficient resource mix is dispatched at any time, including, potentially, consideration of both generator and network assets.

The AEMC has initiated a review to investigate the application of the current system strength frameworks to date and determine whether any improvement could be made to more effectively and efficiently procure and schedule system strength services.16 In addition, a rule change proposal has been submitted by Hydro Tasmania regarding the integration of synchronous services into dispatch.17 These pieces of work will inform the most appropriate way to ensure adequate system strength levels are available in the NEM.

3.3. Considerations for the establishment of a new system service framework

The framework needs to be adaptable to new system conditions, an evolving knowledge of power system requirements and to insights from operating the system at higher levels of non- synchronous generation. The design of the above system services will continue to evolve in the following months and additional services, such as voltage control, could potentially be added to the list. In addition to defining services, the framework also needs to consider how the services can be procured and scheduled to meet system needs. This will be explained in more detail in the next section.

As part of the 2025 project, the ESB is also considering the appropriate procurement structure for the system services, and how this will integrate with investment frameworks. This is in parallel with related pieces of work being undertaken by the ESB and the market bodies for the proposed framework for system services and ahead mechanisms.

16 AEMC (April 2019), Investigation into system strength frameworks in the NEM,https://www.aemc.gov.au/market-reviews-advice/investigation-system-strength-frameworks-nem

17 https://www.aemc.gov.au/rule-changes/synchronous-services-markets

17

4. Framework for procurement and scheduling of system services

4.1. Ensuring resource availability for dispatch

The procurement of resources can be broadly categorised into two timeframes: investment and dispatch. The former is where investment and retirement decisions for an asset are made and typically take place months or years ahead of real-time. The decisions made in the investment timeframe dictate whether the resources are physically available in the system. The dispatch timeframe usually spans from days ahead to real-time, where the resource is scheduled to provide the service. The objective of the market design at the dispatch timeframe is to ensure that:

• Existing resources can be scheduled and dispatched to keep the system reliable andsecure using the efficient resource mix in daily market operation.

• The scheduling process generates the appropriate price signals to incentivise efficientinvestment and retirement decisions in the long term.

For many services, the above can be achieved directly through an appropriately designed dispatch process (both real-time and ahead, discussed in later sections). In this case, providers will make resources available based on price signals that reflect the underlying (or expected) market conditions.

In other cases, there may be significant complexity in scheduling a particular service or market power in sourcing the service, meaning that procuring and scheduling resources to provide those services purely through a dispatch process is difficult. In this case, regulating a framework for contracts could be an effective tool to mitigate these issues.

Contracting directly with the resource to provide the service (similar to NSCAS contracts currently used to address system security gaps) could increase the certainty that the resources will be available when needed by the operator by specifying delivery obligations. Pre-agreeing on the remuneration arrangement could also greatly reduce the likelihood that critical resource providers could exert excessive market power during scarcity conditions. Contracted resources can then be scheduled within dispatch together with other services for the most efficient dispatch in the circumstances.

Figure 1 illustrates the procurement methodology above. The ESB is progressing consideration of the details of this procurement methodology and appropriate procurement and contracting of services and associated considerations such as the mitigating potential market power.

Figure 1 Procurement - from investment to dispatch – illustrative diagram

18

4.2. Scheduling energy and system services in dispatch

An ideal, fully co-optimised dispatch mechanism would schedule all energy and system services, which would capture the interdependencies and trade-offs among them. This would ensure that the most efficient resource mix is deployed to meet all system needs and electricity demand. It would also send the efficient price signals that reflect the true scarcity signal of the resources and incentivise efficient investment decisions. Overall, the efficiency gain would lead to electricity being delivered at the lowest cost and benefit end-consumers through reduced electricity prices. In reality, however, there are challenges to having a co-optimised dispatch mechanism that schedules and publishes prices for all services (whether this is real-time, or inclusive of an ahead dispatch mechanism). Practical limitations such as market power or price formation could mean that forcing a service into a fully co-optimised dispatch might not generate a net benefit. The key challenges are outlined below:

• Scheduling complexity and price formation – Efficient price signals are based onmarginal pricing, which might be incompatible with services that are “lumpy” (for example,the requirement of a service is met only if a specific combination of synchronousgenerators is online).

• Level of competition in market – Effective competition is crucial to ensure that the priceof service is close to its true cost and efficient entry will take place in the long term. Thelevel of competition will be low if the service can only be supplied by one, or only a few participants, in a specific location where the service is required.

• Frequency of service needed – The benefit of incorporating a service into dispatchdepends on the frequency of it being used. Services that are used very rarely and/or onlyin exceptional circumstances (e.g., recovering from a black out) will not benefit from co- optimisation with all other services.

Therefore, different procurement and scheduling methods should be applied to different services based on the extent to which the three challenges above are relevant. It is proposed that there should be three methods:

1. Services that are scheduled and priced within dispatch.2. Services that are scheduled within dispatch, but with pricing determined via regulated

contracts18.3. Services that are not scheduled in dispatch.

Resources that fall into method 1 and 2 can be scheduled within one co-optimisation process. Table 2 outlines the three methods in more detail, with the last column describing the requirements of the features of each service for it to fit into each method.

18 Like NSCAS

19

Table 2 Methods for scheduling (committing and/or dispatching) services

Method Scheduling Pricing Service features

1. Scheduled andpriced in dispatch* via co- optimisation

2. Scheduled indispatch via co- optimisation

3. Not scheduled indispatch

Co-optimised with all other services in dispatch

Co-optimised with all other services in dispatch

Not explicitly scheduled in dispatch, system operatorintervention/instruction ifneeded

Obtained from dispatch

Outside dispatch, potentially based on contracts

Outside dispatch, potentially based on contracts or regulated provision

• Low scheduling and pricingcomplexity, compatible withmarginal pricing, and

• High level of competition, and

• Used regularly

• High scheduling and pricingcomplexity, incompatible withmarginal pricing, or Low level ofcompetition, and

• Used regularly

• High scheduling and pricingcomplexity, incompatible withmarginal pricing, or Low level ofcompetition, and

• Rarely used

*Dispatch in this table, as in the rest of the paper, includes scheduling and commitment in both the real-time and potential ahead market.

Table 3 classifies each service from Table 1 into the scheduling methods based on the assessment against these features. A brief description of how this assessment has been made for the key services is provided following the table.

Table 3 Recommended scheduling and dispatch methods for services

Method Service Scheduling and price formation

Degree of competition

Frequency of service need

1. Scheduled andpriced in dispatch via co-optimisation

Energy Favourable Favourable Favourable

Existing FCAS Favourable Favourable Favourable

Operating reserve Favourable Favourable Favourable

2. Scheduled in

Additional frequency services (incl. inertial response)

Somewhat problematic Favourable Favourable

dispatch via co-optimisation

System strength Not favourable Not favourable Favourable

Voltage control Somewhat problematic

Not favourable Somewhat problematic

3. Not scheduledin dispatch System restart and load

restoration Not favourable Not favourable Not favourable

Based on the assessment above, this study has identified the key system services as discussed in Section 3.2 could be incorporated into central dispatch in addition to existing energy and FCAS products as follows:

• Operating reserves – This service would be both scheduled and priced in dispatch (method1). It is expected to be needed regularly with pricing formation likely to be similar to FCAS,which is well developed in the NEM. It is also likely to have a large pool of suppliers most ofthe time.

20

• Additional services for frequency management – These services would be scheduled indispatch, but further investigation is required to understand if synchronous inertia, ifestablished as a separate service, can also be priced within dispatch (ie, it may fit into either method 1 or 2; further consideration is required). We expect these services to be needed regularly with a large pool of suppliers most of the time. Provision of synchronous inertia, however, could be lumpy as the product is provided by “turning on” the synchronous unit. Whether a potential inertial product can be designed to be consistent with marginal pricing along with energy and FCAS needs further investigation.

• System strength services – This service would be scheduled, but not priced withindispatch (i.e. method 2). While this service is likely to be needed regularly (either suppliedby synchronous generators or condensers), its provision is locational and, like inertia, lumpy.This makes marginal pricing challenging and limits effective competition at times. The specific contracting and pricing methodology will be considered in line with future phases of this work being undertaken by the ESB and the AEMC’s investigation into the system strength frameworks (as discussed in Section 3.2).

As discussed in Section 3.2, the nature of each service needs consideration for how it will be specified and incorporated into the framework at both the investment and dispatch timeframe. Each requires substantial design work, and careful consideration for how it will interact with the physical system to ensure the needs of the system can be met via the proposed market mechanisms. This work must also be completed against a backdrop of new system conditions, an evolving knowledge of power system requirements and against insights from operating the system at higher levels of non-synchronous generation.

There may also be other services identified in the future as needing to be explicitly considered and co-optimised in the dispatch process, such as voltage control and a way to manage minimum operational demand. The framework is designed in such a way to allow for these services to be incorporated as identified and technically defined, with the framework sufficiently flexible to handle the scheduling of services to meet an array of different requirements.

21

5. Ahead mechanism – general design considerations

5.1. The need for an ahead mechanism

The introduction of new markets for system services and incorporating these services into scheduling will likely encourage potential suppliers to provide these services to meet system needs. However, there is still a risk that a real-time only market might not provide the sufficient firming incentive to ensure adequate supply of all system services to maintain security. This is due to some key differences in the engineering and economics between system services and energy supply.

The provision of many system services is considered a public good, as a shortfall in these services could lead to large scale physical disruption in the system. In the event of a shortfall, the system cost could be much larger than the cost suffered by individual market participants who are neither obliged nor incentivised to provide these services. This is distinct from the traditional energy (reliability) provision, where the cost of shortfall (i.e., consumers experiencing involuntary load shedding) is more closely aligned with the loss of the resources that fail to supply during scarcity (i.e., large off-market or derivative contract difference payment for generators who fail to defend a contract position given the spot price for energy will be at the market price cap ($14,700/MWh) in such scarce conditions).19

Many system services are predominantly provided by synchronous units, which are constrained by ramping and start-up lead time to respond to system needs. Synchronous inertia and system strength are two obvious examples, but this is potentially also true for operating reserve until there is further significant expansion of the storage and a fast-start fleet. In a real-time only market, even with separate system service markets defined, non-binding pre-dispatch schedules and prices would be the only signal to notify the markets about potential scarcity conditions. However, resource providers might not respond if there is uncertainty about whether the service will be needed and what the reward will be. If the outturn is a worse-than-expected change in system conditions, the providers might not have enough time to react to provide the system service regardless of the price on the market. Under the current framework, with no market response in the ahead timeframe, AEMO would direct units online to ensure these services would be provided where it was necessary to do so to maintain system security, and intervention and compensation costs would be incurred.

An ahead mechanism (more commonly called an ahead market) is commonly used in electricity markets internationally to provide stronger firming incentives and alleviate the coordination problem in a real-time only market. The coordination problem is likely to be exacerbated by the increased complexity due to the introduction of multiple new system service markets – particularly those that rely on whether generating units are online. In general, an ahead mechanism firms the resource supply and ensures their availability in real-time through a degree of centralised unit commitment and ahead scheduling, while, at the same time, providing market mechanisms to adjust as system conditions change.

Ahead market designs have an ahead run (typically day-ahead in the US) and produce an ahead delivery schedule for resources. Ahead delivery schedules can be designed to be either financially binding or physically binding. Under financially binding delivery schedules, resources can deviate from the ahead schedule in real-time and buy the volume difference at real-time prices. Sometimes it is efficient for resources to produce more or less than the ahead schedule

19 We note that even the alignment of incentives in reliability is not perfect. In practice, the value of customerreliability is more than double of that of the market price cap. Therefore, for every MWh of energy not delivered, the supplier still might not experience the same cost as consumers. There is also anecdotal evidence that many participants have adopted insurance products and effectively de-risk themselves financially without physically backing up the system.

22

based on real-time market conditions, but the financial commitment incentivises appropriate firming for the market. Under physically binding delivery schedules, deviation from the ahead schedule might not be allowed except under extenuating circumstances and could incur civilpenalties – akin to dispatch instructions in the real-time. In either case, the ahead schedulesprovide revenue certainty to providers, who will be more likely to undertake necessary (e.g., fueland staffing) preparation for the upcoming trading days or intervals and are linked to expected physical delivery.

Ahead markets therefore could strengthen resources’ commitment prior to real-time. This can be done via a stronger incentive for resources to honour their delivery schedule (financial commitment), or through the system operator issuing explicit operating requirements that resources must follow in real-time (physical commitment). The latter is similar to the direction mechanism in the NEM, but often implemented in a more efficient way. The operator can physically commit resources ahead of real-time by producing physically binding schedules in the ahead market run, or through committing units out-of-market in some centralised gate-keeping process to fill system gaps not addressed by the market.20

5.2. Additional benefits of ahead markets

While this paper focuses on considerations for an ahead mechanism to drive the efficient dispatch of system security services, ahead markets also aid efficient decision making by resources and aid coordination between markets.

Short-term trading opportunity

An ahead market provides an additional opportunity for short term trading. While the retailers and generators perform most of their risk management through long-term derivative contracts, an ahead market would provide trading participants with an opportunity to adjust their procurement or sale of energy around their long-term trade positions.

Demand side participation

The ESB is developing a design for a two-sided market that aims to price and schedule all the consumption and generation in the NEM.

An ahead market could provide benefits to the two-sided market solution by providing an avenue to enable greater demand side participation for consumers that need to plan consumption decisions in advance. An ahead market may also provide an incentive for retailers to accurately forecast their demand.

Coordination between gas and electricity markets

There is potential for gas-powered generation (GPG) to play an important role during the transition of the NEM by firming the increasing levels of VRE. However, managing increasing levels of GPG will create challenges for both the gas and electricity systems due to the impact that GPG has on gas supply and transportation infrastructure. As such, coordination between gas and electricity markets is likely to become increasingly important.

Peak gas day demand is forecast to grow as residential gas demand in southern states is forecast to outpace declines in industrial demand. There is also the potential for peak gas day demand to coincide with low wind days or generator outages in the NEM, requiring greater amounts of GPG to be dispatched and placing a strain on existing supply and transportation infrastructure.

20 An example is the Reliability Unit Commitment (RUC) process used in the ERCOT market in Texas, US.

23

An ahead market allows a GPG to lock in a revenue stream for the sale of electricity and fund the procurement of fuel to meet its scheduled generation. An ahead market would facilitate timely decision-making, enabling GPG to make nominations for gas supply, transportation and storage prior to commencement of the gas day which supports efficient operation of gas production, pipeline and storage infrastructure.

5.3. Overview of NEM ahead market design

This section describes at a high-level how an ahead market can be designed for the NEM to improve the dispatch process, noting that there are important considerations still to be made in subsequent detailed design stages. The proposed design draws on examples in other jurisdictions that can be adapted for the NEM context. The NEM however, has its own unique characteristics including geographic spread of resources, existing market framework and regulatory arrangements. For example, the NEM has distinct contingency frequency ancillary control products that are integrated with the energy service through real time co-optimisation. Therefore, the proposed ahead markets will be designed to best suit the local context and need of the NEM, rather than directly copied from existing international markets.

The objective of the design of a NEM ahead market is to “firm up” resources and ensure that all system services will be satisfied through the introduction of new system service markets and associated pre-dispatch processes. This includes a stronger commitment mechanism, ensuring all resources critical to maintaining system security can be committed ahead of real-time. It could also be designed to provide stronger incentives for participants to manage variations in output and load to enhance reliability of supply.

The generic ahead mechanism adds an ahead scheduling and unit commitment process to the real-time only NEM market design. (However, not all design options considered have both components. See Section 6 for more details). The ahead mechanism can be designed to co- optimise the output of all resources, capture relevant trade-offs among different services, and enable AEMO to provide a holistic security and reliability “gate-keeping” check to assess resource adequacy and commit appropriate additional resources in the required time. This “gate- check” is included to ensure system security with the changing system dynamic with more complex provision of system services.

Figure 2 illustrates the high-level process description of the key components of a ‘generic’ ahead market for the NEM:

• An ahead market that schedules energy and system services. Participation (i.e., receivingan ahead schedule) in supplying certain critical system services is compulsory, whereasreceiving an ahead schedule for other services such as energy could potentially be optional depending on the detailed design. Similarly, participation by the demand side will depend on the incentives and obligations built into the design. This work would continue to explore the linkages and synergies with the two-sided market work to ensure the most suitable framework can be designed. At the gate-closure for the ahead scheduling run, the most up-to-date bids and offers for all services are co-optimised resulting in a schedule for the services and associated prices, subject to all relevant security and reliability constraints.

• After the ahead market run, AEMO conducts a “gate-keeping” check (called unitcommitment for security, or UCS, see section 6.2) for reliability and security using theresulting schedule (or pre-dispatch information where applicable) and commits additional resources if necessary. This step streamlines AEMO’s current out-of-market intervention process and strengthens the availability of critical resources for system security.

• Ongoing rebalancing takes place in a real-time market. Between the ahead marketscheduling and real-time dispatch, additional intra-day processes can take place at regularintervals. This may be useful given the forecasting uncertainty at the time of the ahead run and would allow both AEMO and participants to make additional adjustments when updated forecasts become available.

24

Figure 2 High-level schematic of ahead process

5.4. Critical reform considerations - commitment design and impact on hedging contract

Centralised and self-commitmentUnit commitment is a major design element in electricity markets. Broadly speaking, the design choices can be divided into centralised and self-commitment. In the former design, participants typically submit multiple part bids (often including start up, min-gen and incremental energy) that explicitly reflect their fixed cost of “turning on” the plant and the variable (marginal) cost of producing additional energy. The system operator then (centrally) decides which resources to turn on in order to meet the system requirement, often over multiple dispatch intervals.

In a self-commitment market design, such as the NEM, participants typically supply incremental- only ($/MWh) bid structure for energy and other system services (such as FCAS in the NEM). To signal their intention to commit, NEM participants typically bid a proportion of their capacity in thelow-price bands (quite often at price floor). In the NEM design, the risk of recovering start-up and min-gen costs is allocated to the market participants. This is as opposed to many central- commitment designs which explicitly allow participants to recover start-up and min-gen costs through “uplift payments” (outside of normal settlement), and the three-part bids are incorporated into the dispatch engine explicitly such that the dispatch centrally decides which units will commit and which will not.

Despite its reliance on self-commitment in the normal market process, the NEM also has a centralised commitment process through direction (intervention). This occurs when there is a reliability and security shortfall and the market response is insufficient. (That is, market participants have not self-committed sufficient resources to address the shortfall) AEMO would then direct resources online and commit them through a centralised decision-making process. Directed participants would be compensated to ensure they recover the cost of being under direction, which is similar to the “uplift payment” in many international markets. Therefore, the proposed streamlining of AEMO’s “out-of-market” intervention process in an ahead mechanism does not introduce materially more centralisation of resource commitment compared to the current NEM “out-of-market” direction process.

25

Whether additional levels of centralised unit commitment would be needed in the “in-market” ahead or real-time scheduling process is still to be determined. It may be possible that the incremental-only bid design could be retained in both the ahead and real-time schedules for energy, FCAS and other additional system services (where applicable). In other words, resources could continue to signal their willingness to self-commit in response to market conditions by bidding part of their capacity into the negative price bands. Whether the self-commitment design continues to be appropriate with the introduction of additional system services and an ahead mechanism could be further explored with stakeholders. While some system services21 might have gate-closure before real-time or be subject to a physical delivery schedule in some design options, it needs to be recognised that such a design is intended to avoid the need for direction/intervention, which in itself is a form of centralised commitment. The crucial design consideration is to find the right balance so that the right mix of resources are committed at the right time to ensure a secure and reliable system at the lowest cost.

Financial contract marketFinancial hedging contracts play an important role in the NEM. They help participants to manage their risks of exposure to volatile energy prices and provide some revenue certainty to underpin new investments and medium-term availability and fuel decisions. Most standard (i.e. swap and cap) contracts are referenced to the regional reference node and are settled against real-time pool prices.

The introduction of an ahead mechanism, to the extent that it contains some financially or physically binding ahead schedules, would introduce another basis against which financial contracts could be referenced. A change to reference contracts against the day-ahead price would cause a disruption to the financial contract market and would require appropriate transitional mechanisms, like those currently planned for the Five Minute Settlement rule change and the proposed Locational Marginal Pricing in the COGATI reform, to be put in place. As the NEM currently only has hedging contracts for electricity prices, ahead schedules for system services are unlikely to have any direct impact on them. As described in Section 6, there is a spectrum of ahead design options considered, and many of these either do not have an ahead energy schedule or allow participants to take up the ahead energy schedule on a voluntary basis. In these cases, the impact of the ahead mechanism on the financial hedging markets will likely be small. If it is compulsory for participants to take up ahead energy schedule (as in the option described in section 6.6), its impact on an existing hedging contract could be material. However, in these cases, the basis of hedging contracts could be moved to ahead, rather than real-time prices, and transitional mechanisms could be put in place at the early implementation stage to facilitate this process.

One potential concern is the impact of “uplift” payment on the financial contract market as the costs cannot be hedged by customers. However, uplift payments already exist in the current NEM under the direction mechanism, so an ahead mechanism is unlikely to cause more uplift payments through its out-of-market intervention. Further, as discussed above, the design of a NEM ahead mechanism could potentially retain the current self-commitment design where the risk of cost recovery continues to be allocated to market participants. In this case, the uplift payment to “make-whole” participants for their normal in market operation might not be needed. Therefore, it appears that the NEM ahead mechanism could be appropriately designed to minimise the risk of uplift payments, and this will be considered further in the next step of designing the framework.

As the ahead market is part of the wider post-2025 market design project, its impact needs to be examined in the context of other potential reform pieces such as locational marginal pricing ortwo-sided market design. Each of them could have a material impact on the existing regional-reference-node and real-time based financial hedging market. In many international markets,participants can also trade financial transmission rights (FTR) or arbitrage between real-time and

21 For example, services that are predominantly provided by synchronous generators that have long start up times.

26

ahead energy prices (e.g., through virtual trading) before real-time, which could further increase the liquidity of such instruments and aid price discovery.

5.5. High-level design trade-offs

Introducing an ahead mechanism requires consideration of a new set of compromises to determine the most efficient solution while securing the system. If more resources are committed before real-time, there may be less options available to respond to new market conditions. For example, demand for some services could turn out to be lower than expected after the ahead scheduling and commitment decisions, in which case the resources are over-committed, leading to higher cost to end consumers. On the other hand, without enough resource commitment in- time, the market might not have enough means to address an unexpected gap in system, leading to large supply disruption and potential system failure, which would also be costly to end consumers.

Ahead market commitment can be seen as buying an insurance premium to protect the system against large-scale disruption caused by unexpected changes in market conditions. The optimal ahead market design is about allocating resources efficiently in an uncertain environment to ensure that risk is managed within an acceptable level at lowest cost. An efficient solution to uncertainty cannot rely solely on committing all resources ahead of time when the real-time need is still largely unknown, or leaving everything to the last minute only to find there is not enough resources to meet the demand. Instead, it requires the appropriate utilisation and deployment of the slow- and fast-start generation and storage fleet at different operational timeframes, potentially from day-ahead to real-time, based on their operating characteristics and the system needs.

Efficient ahead market design therefore needs to balance the fundamental trade-off described above. This requires the design to carefully balance the following aspects:

• Lead-time of ahead decision - This includes how far ahead before real-time decisionsshould be made and the number of real-time intervals the ahead decisions should cover.Scheduling and committing too early means that decisions are made with a large degree of forecasting error but leaving everything too late could run the risk of not having enough resources to cover unexpected gaps in the system.

• The level of ahead commitment – This concerns the amount of resources that should becommitted, and the form of commitment. Having more resources committed increases thelikelihood that system needs will be met. However, committing too many resources tightly to an inflexible schedule means that they will have limited options to respond to changes in market conditions, leading to higher dispatch cost. This is dependent on detailed design decisions to determine the appropriate level of procurement or commitment of provision of the various services included in the framework and the associated trade-offs.

• Number of binding ahead processes – this concerns the gate-closure rules which couldvary for different resources or services, and relates to both of the aspects outlined above.Multiple commitment times allows for greater flexibility to accommodate the physical requirements of the resources and power system. For example, a service that is predominantly supplied by synchronous resources might have an earlier gate closure compared to a service that can be supplied by all resources including VRE and battery. However, the above benefit also needs to be weighed against the increased complexity in the scheduling process.

For brevity, this report will not always make explicit reference to the above issues when presenting the four ahead design options in the next section. However, these trade-offs are fundamental to all ahead mechanisms to be investigated and evaluated. Importantly these also need to be considered against the potential risk and inefficiencies in leaving all decisions to real- time where there may be reduced options available to secure and maintain the reliability of the system.

27

6. Ahead mechanism design options

6.1. Overview of spectrum of options

This section provides a more detailed description of the four design options that have beenspecified to describe the spectrum of potential design approaches, as shown in Figure 3. Acommon element among all options is that they all have a physical unit commitment ahead ofreal-time (a unit commitment for security, or UCS). Another common element is that system services markets are established, as were discussed in Section 4 and 5. This implies pre- dispatch would incorporate those system security services, with relevant offers and scheduling information taken into account. Additional consideration is required, and will be completed as further work is undertaken, for how all existing processes, including for example, pre-dispatch, dispatch, forecasting, bidding, and settlement will integrate with the proposed new services and adaption to the dispatch mechanism.

The options for the ahead mechanism differ based on the level of co-optimised market scheduling ahead of real-time, the potential of commercial participation in the ahead process and the level of mandatory participation in the ahead scheduling. Going from left to right in the figure, the scheduling requires greater degree of mandatory participation (i.e., receiving ahead schedules), includes more services, and is based on greater level of co-optimisation.

Figure 3 Design options overview

There is a myriad of possible solutions within the spectrum presented, with each option potentially consisting of sub-variants. A preliminary and high-level description of how each of these options would work is described in this section. Before presenting each individual option, Section 6.2 describes the UCS process, which is a key element in all options. Further detailed analysis is needed to define design issues and evaluate the benefit and cost of each option. Many design details will also depend on or need parallel consideration for the outcome of other major market review and reform processes currently underway. This includes:

• The AEMC’s Coordination of Generation and Transmission Investment (COGATI) review,which is investigating the potential of implementing locational marginal pricing within theNEM and related financial products for managing transmission congestion.

• The ESB’s two-sided market reform project, which is developing a design for the pricingand scheduling of all consumption and generation in the NEM.

• The AEMC’s investigation into system strength, which is looking at amendments to theexisting minimum system strength and do no harm framework.

28

6.2. Unit Commitment for Security (UCS) process

The purpose of the UCS process is to add an enhanced centralised commitment design facilitated by better information provision and a more integrated optimisation process that ensures critical resources will be available to deliver secure and reliable electricity supply in real-time.

The need for enhancement to the NEM commitment mechanismThe ongoing process used for the directions in South Australia for system strength has demonstrated that the current commitment and intervention design and compensation methodologies are not optimised for a system with increasing system security shortfall risks. The current direction process is ad hoc as AEMO’s direction relies on an identification of gaps based on pre-dispatch offers that can change at any time and requires specific collection of information each time to understand intervention options.

The current process is also incremental as AEMO’s direction can only commit the specific generating units that are required to resolve the identified gap. Under the new intervention pricing rule post December 2019, direction for services that are not traded on the market, which includes system strength, no longer triggers intervention pricing and compensation for affected participants. As the additional energy from directed units will likely reduce energy prices, other synchronous generators might prefer to go offline, causing further system strength gaps and forcing AEMO to issue additional directions. While such instances will potentially be reduced with the establishment of new frameworks to secure system services, the increased complexity of meeting multiple system security needs together with energy, combined with the declining availability of synchronous resources to supply some system services, will likely increase the risk of some critical resources being unavailable in real-time.

While the operator endeavours under the current mechanism to minimise the costs of directions needed to fill the security and reliability gaps as they arise, decisions must generally be made in a short timeframe, based on estimated or manually obtained cost information. The decision process is not co-optimised and cannot fully consider the economic impact on dispatch for the entire system. The current process does not give the system operator sufficient confidence that all resources will be available in real-time and exposes both directed participants and the rest of the market to greater operational and financial risks.

The current intervention process is appropriate in a market environment where shortfall of system security and reliability is rare, and the primary source of the occasional market failure is shortage of generation to meet electricity demand. However, a new enhanced commitment mechanism is required now that the potential for system security and reliability shortfall is higher due to the declining availability of resources that can supply certain system services and the increasing complexity of scheduling for multiple services.

UCS processThe UCS process is shown in the flow chart in Figure 4, with each step described below.

Figure 4: Unit commitment for security process

While the UCS process is designed to run at defined time frames to improve the effectiveness of the current intervention process, AEMO would still have the capability to issue an ad hoc

29

intervention outside the process if an unexpected system gap arises. However, the implementation of the UCS process will likely greatly reduce the need of such ad hoc directions.

1. Participant pre-dispatch information provisionThis step is similar to the current pre-dispatch process where participants will submit bids into thepre-dispatch, as well as relevant physical characteristics of their plant.

Participants will be required to provide:

• Bids - As now, bids will be for real-time energy and of each defined system service(currently just FCAS, but to be expanded to include additional system services). Where aservice has been contracted for, this may be required to be reflected in the resource’srelevant bids.

• Operating schedule – The relevant physical characteristics of their plant (like ramp ratesnow), will be expanded to include information regarding capability of their resource toprovide each service and the physical characteristics and limitations that are relevant tothe assessment of system security and reliability. Examples could include on/off decisions, running mode (e.g. if the resource can run in generation or synchronous condenser mode), storage capability and maximum available capacity for each service.

• Economic cost and operating information – Economic cost and operating informationsuch as start-up cost, minimum generation level and cost, incremental generation cost,cost of running in other modes (e.g., synchronous condenser mode), minimum on/off time, time to synchronise (which would include the outage recall times if it applies) and other relevant cost and plant information for providing system services.

Bids and operating schedules will be supplied at the pre-dispatch required granularity (such as five-minute or half-hourly). As new system services are introduced, bids and additional relevant physical plant information (if any) will also be included.

The additional economic cost and operating information would only be used by AEMO to determine the least-cost out-of-market commitment should a security or reliability gap be identified. It would not be used to determine plant dispatch or market prices. The resultant impact on market prices and appropriate compensation mechanisms are important issues for future consideration as part of the design development and are discussed in the relevant sections below. Sensitive commercial information would be supplied on a confidential basis to AEMO. This information provision replaces current manual processes used to collect information when directions are required.

2. AEMO security and reliability assessmentDuring pre-dispatch, AEMO assesses system security and reliability in the forward period and identifies potential shortfalls. The assessment will be based on the most up-to-date pre-dispatch information and most recent forecast of system conditions such as demand, weather, network configuration and any other system limitations. Forecast shortfalls would be communicated to the market, like current Lack of Reserve notices, to allow for a market response right up to a pre- defined time when any shortfalls will be met by the unit commitment process.

3. Unit commitmentAt pre-defined times AEMO would commit additional resources (which could be supply or demand) to address any security or reliability gaps that remain for the relevant assessment window (i.e. AEMO would run the UCS process).

30

For the purpose of the rest of the document, we will collectively refer to additional resources committed by AEMO at this step as “committed out-of-market”, and other resources that made themselves available as indicated by the pre-dispatch outcome as “commercially-committed”.22

The pre-defined lead time ahead of real-time for the unit commitment run needs to be carefully designed. This is because a shorter lead-time means that AEMO and the market participants have more precise forecasts of system and plant information for any assessments and consequential commitments, if any. On the other hand, a lead-time that is too short reduces the operational flexibility and there may not be enough resources (or only expensive resources) to deploy to fill the gap, as some resources (such base load generators and demand response) might take a longer to activate.

With the expanded pre-dispatch and dispatch process to extend to additional system services markets, gaps in system services will be more easily identifiable prior to the unit commitment run.

The additional resources committed could be resources that normally participate in the market, but have not made themselves available, are on outage but could be recalled, or from emergency reserve providers. AEMO would commit resources out-of-market using a least-cost approach based on economic cost and operating information supplied during pre-dispatch. For emergency reserve providers, the terms of their service contract would also be taken into account at this stage. If multiple security gaps are identified, AEMO would endeavour to co-optimise across them (i.e. fill all gaps at least cost overall).

4. Publish Physical Commitment Plans (PCP)The PCP is the key output of the UCS process and specifies a physical commitment schedule for relevant resources. The PCP is a binding physical commitment schedule that limits the operation decisions of applicable resources. The PCP places requirements on certain aspects of a resources’ operating characteristics such as synchronisation and de-synchronisation time, running duration, restrictions on the level of output, etc. Resources committed out-of-market will always be subject to PCP, whereas commercially-committed resources would potentially be subject to PCP (see below).

5. Physical operationIn order to maximise the flexibility of participants to respond to real-time market conditions, the PCP aims to place the minimum level of commitment only on those participants that are pivotal for system security in real-time. A physical commitment schedule can take many forms and does not necessarily mean that participants cannot change the level of output at all for real-time operations.

• Resources committed out-of-market in the previous step would receive a PCP whichoutlines the operating parameters they must follow in real time.

• Some commercially committed resources could potentially be subject to a PCP if AEMOconsiders that they are pivotal to system security in the relevant real-time. This means thatcertain changes in these resources’ operating schedules could cause a system gap, leading to further intervention by AEMO. For example, de-synchronisation of a commercially committed resources could potentially cause a shortfall in inertia or system strength. The PCP will require that these resources do not change those key

22 The terminology choice here recognises the fact that additional resources committed at this step have notresponded to market signals (for whatever reason) and hence triggers “out-of-market” intervention by the operator. Other resources are made available to the system by their owners for commercial reasons in response to market signals.

31

characteristics without explicit permission from AEMO which would take into account the reason to decommit.23

The proposed PCP on commercially committed resources trades-off their flexibility to respond to new market conditions versus the security of the whole system. Imposing overly restrictive physical commitment would weaken participants’ incentive to respond to market signals and eventually undermine the effectiveness of the market. It is therefore crucial to ensure that the PCP on commercially committed resources are as light as possible. The above design principle recognises the fact that had the commercially committed resources changed certain aspects of their operating plan, the system would have a gap causing AEMO to intervene (meaning the resource is pivotal). The PCP would be placed on the relevant critical aspects of these pivotal resources to avoid unnecessary burden on control room operation and minimise risks to the system. The resources could still change the relevant operating characteristics under the PCP after getting explicit approval from AEMO.24

In practice, the PCP for commercially committed resources can be expected to be primarily placed on the synchronisation and de-synchronisation times, as these are the most crucial characteristics for supplying many system services. The resources would still be free to re-bid other operating characteristics as long as it does not violate the PCP (for example, changing maximum availability while remaining online, or synchronising earlier).

The PCP commitments would be enforced in a similar way to the existing directions process: resources committed under a PCP would face civil penalties for deviating from the plan.

While the UCS is proposed to be a standardised process that will periodically check for system gaps, it is expected that out-of-market commitment would be used relatively rarely when all relevant system services are properly valued in the market and incorporated into an appropriately designed dispatch process that provides the efficient firming incentive for them.

Issues for consideration

Daily and hourly UCS process (DUCS and HUCS)The design can consider two types of UCS. The first UCS covering the trading day is called Daily- UCS (DUCS). As noted in section 6.1, the ahead mechanism design does not have to be based on daily scheduling and could instead run for a block of several hours. However, the report will use the term DUCS and trading-day here for the sake of presentation. The DUCS will assess system security and reliability and identify any gaps for the entire trading-day and commit resources. When operating within the actual trading day, hourly-UCS (HUCS), which can be run every hour or once every several hours, will check for any new gaps and commit further resources, if necessary, for the remainder of the trading day.

The combination of DUCS and HUCS gives the operator more flexibility when committing resources. For example, in the DUCS process, it could primarily commit resources that have long start-up time to address the most critical system gaps and wait until the subsequent HUCS process to commit additional more flexible resources if the need persists.

Intervention pricing and compensation mechanismsIntervention pricing and compensation mechanisms are a key factor accompanying the operator’sout-of-market commitment actions. The purpose of these mechanisms is to ensure that the out-of-market commitment by the operator does not undermine the original scarcity price signal,

23 As with any new process, the appropriate compliance frameworks and associated overheads will be an importantelement to consider in the evaluation of the options.

24 This could happen under a change in market conditions or availability of other plant causing the resource to be nolonger pivotal.

32

which is crucial for incentivising long-term investment. The operator’s intervention in the market affects:

• The resources directly committed, which need to at least recover the cost incurred as theresult of the intervention.

• Other resources not directly committed by the operator, but which are still affected by theintervention due to changed dispatch pattern and spot prices.

The appropriateness of the NEM intervention pricing framework needs to be reviewed to ensure it remains fit-for-purpose with the UCS. This also includes the compensation process for resources committed out-of-market. Given the expected introduction of the new service markets, it is likely that intervention pricing can be applied more widely in the case of AEMO out-of-market commitment in the UCS, which will likely improve the efficiency of the market.25 However, given the establishment of distinct services markets, it is unclear whether resources directly committedout-of-the market (i.e., directed participants) should continue to be remunerated based on historical energy market prices. These issues, including their impact on participants’ ability to manage their financial hedging contract positions, need to be further considered, and will also take into account relevant reviews into the matter, such as the AEMC’s recent investigation into invention mechanisms.26

Finally, the current UCS design envisages that AEMO might impose PCP on some commercially- committed resources that are pivotal to system security, who would then need AEMO’s explicit permission to deviate from their PCP. Whether intervention pricing and compensation mechanisms should be applied if such resources’ permission to deviate is denied by AEMO also needs to be considered in the design.

6.3. Option 1 - UCS-only option

Objectives and rationaleThe “UCS-only” option seeks to mitigate security and reliability shortfall risks by introducing the UCS as a formal commitment process in daily market operation. The introduction of the UCS will improve information provision to AEMO and streamline the out-of-market commitment process to address forecast gaps in power system security and reliability. It also reduces the risk to the system by unexpected changes in operations from resources critical to system security.

While pre-dispatch will likely need to be enhanced to incorporate new system services and facilitate information exchange between AEMO and participants (in both directions), there is no binding ahead scheduling and commitment, except for any physical commitment for the delivery of system services applied by the UCS process through the PCP for relevant resources. As such, this option represents the minimal departure from the current NEM design.

Overview of the processJust like in the current NEM, participants supply pre-dispatch information including price and quantity offers for all energy and system services over a forward window, while AEMO forecasts

25 As of December 2019, the AEMC has made a new rule that intervention pricing and compensation for affectedparticipants will only apply to “traded-services” on the market. In the current NEM this means energy and FCAS only. Intervention caused by shortage of system strength no longer triggers intervention pricing until a market for this service is established.

26 AEMC (August 2019), Investigation into invention mechanisms and system strength in the NEM,https://www.aemc.gov.au/market-reviews-advice/investigation-intervention-mechanisms-and-system- strength-nem

33

the energy market demand/supply balance.27 The pre-dispatch information remains generally not binding, and participants are able to change their offers and bids unless they are subject to a PCP from the UCS processes.

The process of the UCS-only option is illustrated in Figure 5. The key feature of this option is the UCS as explained in detail in 6.2, which is a checking process for any system service gaps and would commit additional resources out-of-market to if such gaps are found and produces a PCP that sets out physical commitment schedules (if any) on relevant resources. As discussed before, there can be a combination of daily and hourly UCS (DUCS and HUCS) to both allow AEMO more options to respond to new gaps and more flexibility to commit fast-start resources closer to real-time. Participation in pre-dispatch and the UCS process (i.e., being subject to PCP) is mandatory.

Figure 5 Ahead process – UCS-only option

*Note: P in this diagram represents “price”, Q represents “quantity”

Issues for considerationThe design issues related to the UCS process itself have been discussed in the previous section. These are applicable to all options. This sub-section therefore discusses the issues associated with introducing only a UCS.

Market interactionsThe effectiveness of the UCS also depends on the wider market it interacts with. The out-of- market commitment through the UCS (or pivotal commercially-committed resources seeking to deviate from PCP but not approved by AEMO) should not happen on a regular basis in a well- functioning market, in which the appropriate price signals and firming incentives ensure resources are made available when needed in market (e.g. through the to-be established system services markets).

Given the increasing variability and uncertainty in the NEM and the complexity of scheduling multiple services, there could be instances where resources do not make themselves available to meet system needs. This could arise due to occasional coordination failure, participants’ revenue uncertainty from provision of certain services, or misalignment of participant’s commercial

27 AEMO publishes pre-dispatch (PD) information on a half-hourly granularity during 16-40 hours ahead; and on a 5-minute granularity during the 1 hour before dispatch.

34

incentives with the need of the system. In order to prevent out-of-market commitment from being a regular occurrence, the design needs to ensure that system services markets appropriately translate the physical needs of the system into transparent market prices, and/or consider strengthening the firming incentive through additional ahead scheduling mechanisms (as discussed in the following options).

6.4. Option 2 - UCS plus Voluntary Forward Market

Objectives and rationaleThe UCS plus voluntary forward market option is introduced as a mid-way point between a commitment-only ahead design and a formal day-ahead market. This design has a voluntary short-term market where participants can trade among themselves for energy or with the system operator for system services. Participants with voluntary forward market obligations are likely to be firmer in their real-time supply due to the potential pecuniary impact of non-delivery. They will likely to make better short-term preparations for the plant including short-term fuel procurement, staffing decisions, etc. The voluntary forward market obligation therefore increases operational certainty to both the participants and the system operator. This may increase the commercial availability of the resources in the real-time and potentially reduce reliance on AEMO undertakingout-of-market commitment in UCS. The voluntary forward market for energy and system securityservices may therefore increase the reliance on in-market response to resolve system needs.

OverviewThis option adds an additional voluntary forward market component to the UCS-only option, as illustrated in Figure 6 Ahead process - UCS plus voluntary forward market. The voluntary forward market is a voluntary participation trading process and can be designed for energy trading only or could cover other system services. It is proposed that participants supply trade information on unit instead of a portfolio basis, which will increase transparency to AEMO and facilitate operational planning and system monitoring. Depending on the design and the services traded on the voluntary forward market, it could take place either before or after the UCS process (or both).

Figure 6 Ahead process - UCS plus voluntary forward market

*Note: P in this diagram represents “price”, Q represents “quantity”

The voluntary forward market process

The design of the voluntary forward market has been based on AEMO’s recent rule change proposal for a Short Term Forward Market (STFM), with relevant adaptions to integrate with a

35

new framework for system services and with a USC process.28 The ESB notes that the AEMC has made a recent draft determination not to proceed with the STFM in the current framework.

Voluntary forward market for energyThe voluntary forward market for energy can be designed as either a centralised or decentralised trading platform.

In the decentralised variation, there will be no central market clearing process or centrally awarded energy price or quantity schedule. Instead, market participants submit their bids and offers for energy on a common transparent platform. A trade is formed when a seller’s offer matches a buyer’s bid. Gate-closure might not be needed in this design and participants can trade right up until real-time.

Alternatively, energy could also be traded via centrally cleared periodic auction where all bids and offers are settled by the system operator at a defined time at a common (regional or locational) clearing energy price. Liquidity will be crucial for efficient price discovery, and the design needs to ensure that demand will participate in the trade.

Voluntary forward market for system servicesThe voluntary forward market could also be designed for trading some system services. As most system services are “public goods” that benefit the entire grid, the demand for these services is likely to be determined by the system operator (similar to the current procurement of FCAS). While the design could depend on the nature of the system service traded, the operator could procure some “base” or “minimum” level before real-time. Alternatively, the procurement could be based on forecast system security gaps (such as system strength). Through the voluntary forward market both the system operator and the participants will receive a firm schedule for delivery of system services before UCS and real-time operation.

Transactions on the voluntary forward market related to some system services might need to be physically binding and be visible to the system operator to facilitate the UCS commitment processes.

Issues for consideration

Ensuring physical feasibility of tradesIn order to improve the reliability and security of the system, trades on the voluntary forward market could be restricted to those that are physically feasible. For example, every trade on the voluntary forward market could be reconciled against the pre-dispatch information (i.e., participant’s operating schedule). Trades will be disallowed if it is not physically feasible based onpre-dispatch information such as physical status of the units or maximum available capacity. Animplication of this feature is that virtual trades will not be allowed on the voluntary forward market. Therefore, benefits from improved transparency need to be weighed against reduced liquidity and price discovery in the voluntary forward market.

The buyer of system servicesUnlike energy, it is likely that the total demand for system services can only be effectively determined by AEMO. It flows then that the buyer for system services is likely to be AEMO, on behalf of the system, that is, it will be a one-sided market. There are alternative options whereby AEMO could determine the level required but the obligation for procurement could be smeared

28 AEMC (December 2019), Short Term Forward Market, https://www.aemc.gov.au/rule-changes/short-term-forward-market

36

among the participants. Further work is required to understand the implications of this approach for the final design.

Interaction between the voluntary forward market and UCSAs the UCS commits additional resources out-of-market, it should take place after the formal market process to reduce the impact of intervention. However, having gate-closure for the voluntary forward market might limit the opportunity for participants to trade in response to new market conditions. Whether voluntary forward market could take place after the DUCS might depends on the centralised/decentralised design and the services traded. Potentially a decentralised voluntary forward market that trades energy only can take place both before and after the DUCS. Centralised voluntary forward market design for energy or a voluntary forward market for system services will likely need to be cleared before the DUCS process.

Identifying the benefit of additional firmingThe extensive use of derivative forward contracts (e.g., swap and cap) are typically seen as providing firming incentive to suppliers through financial commitment, although recently there is a trend of participants seeking to cover their risk through financial risk insurance products that are not linked to physical assets. Further investigation is needed to understand the additional firming benefit of a voluntary forward market that trades energy. On the other hand, currently there is no forward financial market for other system services. Therefore, a voluntary forward market for system services is expected to improve the ahead procurement of these products.

6.5. Option 3 - System security ahead market

Objectives and RationaleThe “System security ahead market” option seeks to mitigate system security and reliability shortfall risks at lowest cost by introducing a fully-fledged day-ahead market that integrates the provision of energy and other system services. This option aims to ensure that AEMO has greater visibility and can dispatch all resources that can provide system services ahead of real-time, if required. This option also seeks to improve the commercial availability of resources and produce the most efficient dispatch outcome through co-optimising all energy and relevant system services in the ahead scheduling.

As with option 1 and 2, all participants must bid into pre-dispatch for energy and system services. In this option, providers of critical system services29 will receive a physically binding ahead schedule, but schedules (physical or financial) for other services including energy will be taken up on a voluntary basis.

Producing an ahead schedule based on the full co-optimisation is the key distinction between Option 3 and the previous “UCS plus voluntary forward market” and “UCS-only” options. With the expected further decline in energy and storage cost but increasing scarcity of system services associated with a dwindling availability of synchronous resources, there is likely to be greater benefit in improving the scheduling efficiency of non-energy services. The ahead schedules will also improve the firmness of supply through the combination of mandatory binding physical schedules for critical system services and voluntary physical or financial schedules for other services including energy. This will reduce the risk of a system gap hence lower the reliance onout-of-market commitment in the UCS. The efficiency gain through co-optimisation and the greater reliance on in-market response to system needs will likely lead to overall lower costs of delivering electricity and benefit end consumers through more affordable energy prices.

29 Exactly which system services will be considered critical will be investigated in the next phase.

37

OverviewThe system security ahead market, illustrated in Figure 7, introduces a full co-optimisation of all energy and relevant security services in the ahead scheduling process. In this option, all registered participants must bid into pre-dispatch for all energy and system services (where relevant), as per the requirement in option 1 and 2. When the ahead scheduling takes place, at apre-defined time ahead of real-time, it would use the most recent pre-dispatch bids and AEMO’s system forecast information as inputs. This ensures that the scheduling reflects the most recent (expected) demand and supply information for the entire system, including all energy and system services. The ahead scheduling mechanism would then publish ahead schedules for energy and system services by fully taking into account their interdependencies and trade-offs. For system services that are considered critical, their providers will receive the ahead physical schedules for these services on a mandatory binding basis. For other services including energy, their providers will receive the (physical or financial) schedules for them on a voluntary basis (that is, if they have opted to be bound by this schedule). Therefore, participants have the choice of not receiving an ahead energy schedule and leave their energy market revenue solely to the real-time balancing market instead, although their pre-dispatch energy bid would be used as an input in the ahead scheduling. Further consideration is required to understand how two-sided market principles will integrate with the ability to take a voluntary ahead schedule where this is applicable.

The DUCS takes place immediately after the ahead scheduling. There will be tighter linkage between the ahead run and the DUCS process, as the ahead scheduling outcome becomes an input into the DUCS process.

Figure 7 Ahead process – system security ahead market

*Note: P in this diagram represents “price”, Q represents “quantity”

Issues for consideration

Efficient participation in the energy ahead marketWhile energy ahead schedules are taken up on a voluntary basis, being subject to an energy ahead schedule could give participants additional incentives to improve the accuracy of pre- dispatch bids. Therefore, higher participation in an energy ahead market would improve the co- optimisation benefits via more accurate pre-dispatch information. This is in addition to the resource firming benefit as a result of the ahead financial commitment incentive. The design therefore should seek to incentivise, or at least not discourage participation by energy resource providers. It is important to note that the design objective is to achieve the efficient level of energy

38

participation in the ahead market recognising that some residual volatility will exist in the real-time market.

Given energy can also been seen a “by-product” of providing some system services, it is possible that as system service providers will receive ahead schedules that are physically binding, there might be a natural incentive for them to participate in the energy ahead market in order to lock in an advantageous ahead energy price that at least covers the level of “by-product” (e.g., min-gen) energy production.

Demand (price-responsive or not) is also another important group of participants in the energy ahead market. Broadly speaking, this two-sided participation could be done via:

• Retailers or load customers directly taking an ahead energy schedule30

• Facilitating demand response providers trading ahead of real-time within the currentdesign

Direct participation in this manner by retailers is consistent with that in existing facilitated gas markets. Participants in the Victorian Declared Wholesale Gas Market (DWGM) are required to provide demand forecasts that are profiled across the day for their uncontrollable customers (e.g. household and commercial users), and this sets their price-taker bids. Similarly, participants in the short-term trading market (STTM) hubs are required to bid for the demand of their retail load.

Multi-interval optimisationThere are broadly two design choices in terms of optimisation window in the scheduling (ahead or real-time).

• Single interval optimisation where the dispatch intervals are optimised individually andseparately from each other31

• Multi-interval optimisation, where the ahead market is co-optimised via one single co-optimisation that spans all relevant dispatch intervals

Given the ramping constraints and the lead-time to start required by some resources, as well as the energy limits of electricity storage systems (particularly batteries), it seems that explicit intertemporal trade-offs might need to be incorporated to some degree to improve the efficiency of the schedule. One risk of multi-interval optimisation is that it could over-ride participants’ own commercial and risk preference and could cause conflict with their strategies to defend derivative contract positions32. Such issues might potentially be addressed through resource changing their bidding strategies (bidding low price to ensure its contracted volumes are dispatched).

Impact on financial hedging contractCurrently forward contracts in the NEM are linked to the real-time wholesale spot market, as this is the only market. With the introduction of an ahead market, it is possible that the basis of forward contracting might move to the ahead market. This is a common practice in international markets. It is also likely that the incentive to participate in the day-ahead energy scheduling is higher the more forward contracts that are based on the ahead market. Appropriate transition

30 The ESB is currently investigating a two-sided market design for the NEM, which is aimed at facilitating demandside participation..

31 This means the objective function will only incorporate the total resource cost incurred in this interval. However,the optimisation still needs to be done sequentially, as the solution of interval t-1 will be the initial condition for interval t.

32 For example, a battery provider might want to generate 100 MW in interval t due to its current cap contract level.However, such preference or offer might be over-ridden by an inter-temporal algorithm as it might reduce the output of the battery in interval t below 100 MW in order to preserve its storage for a later forecast peak, which might or might not realise due to uncertainty.

39

would be part of the implementation considerations to allow the derivative contract market appropriate time and mechanisms to adapt to any introduction of an ahead market.

Level of procurement of system servicesDepending on the nature of the services and the amount of competition in the market, the level of procurement of different resources in the ahead market needs to be carefully considered.

The demand for some services such as operating reserve could vary significantly in the short- term. Buying too much in the ahead market (and imposing physical schedule on resources) could offer greater certainty but might also lock in early forecast error or uncertainty, which would otherwise reduce closer to real time. On the other hand, buying a smaller amount in the ahead process could reduce the likelihood of over-procurement, but potentially has the risk that a real- time gap could only be supplied by very expensive resources or no resources at all.

Depending on the nature of the services and the amount of competition in the market, the level of procurement of different resources in the ahead market needs to be carefully considered. In principle, AEMO should purchase less in the ahead timeframe if a service can be supplied by more resource types, including some that are flexible and available with little notice; and where there is large ahead uncertainty regarding real-time demand. The converse is true for services that are provided by resources that need an earlier indication they will be required (eg. have a longer start-up time) and there are less options available to procure the service closer to real- time.

This is an important factor for consideration when designing the scheduling for specific services as they are introduced. Again, there are well-established practices internationally that can be drawn on and should be considered for their applicability to the NEM design when the specific system services are introduced. The introduction of an ahead market together with the real-time market introduces a level of flexibility in the level of procurement at various stages, which could also adapt as system needs change and technology evolves.

Schedule adjustment after ahead marketSchedules made in the ahead market will inevitably contain inherent uncertainty given they are based on a forecast. As variability and uncertainty increase in the market, so does the forecast error in ahead schedules. Therefore, the design could potentially consider options of allowing adjustment of the ahead schedules to reflect the new market dynamics. In this case, the operator can publish updated intraday binding schedules at predefined timeframes for the remainder of the trading day (or the remainder of the real-time period to which the original schedule applies). However, the potential settlement complexity arising from multiple ahead positions referring to the same real-time interval needs to be understood. The inclusion of intraday binding schedule adjustments does not impact the continued publication of pre-dispatch schedules approaching real-time. In this option, a particular pre-dispatch schedule would be used to define the “ahead market” outcomes, pre-dispatch schedules would then continue to be published as “deviations” from the ahead market to indicate the real-time physical dispatch needs.

6.6. Option 4 – Compulsory ahead market

Objective and rationaleThe compulsory ahead market seeks to reduce the security and reliability risk in the system by further enhancing firming and commitment in the ahead process. While the system security ahead market design (option 3) focuses on ensuring the physical delivery of key system services that directly impact system security, it still potentially leaves considerable freedom for participants to change their level of energy output between pre-dispatch and real-time. Sudden unexpected reduction in energy generation (e.g., unexpected ramp in wind or solar availability) in real-time could lead to insufficient supply to meet energy demand. This can cause both security (e.g. frequency deviation due to higher demand than supply) and reliability (e.g. involuntary load shedding to keep power balance) issues. This option places tighter obligations on participants

40

compared to option 3, and a number of variants for how to do this are presented in this discussion and in the next stage will be evaluated for how they may be appropriately designed to mitigate the emerging risks in the system operation.

They key difference in this option is to require mandatory all participants to receive an ahead schedule for all energy and system services markets. Load will also be required to participate in the ahead process by receiving an ahead schedule for energy and other markets they could provide services in. This option could also consider more stringent gate closure requirements so that participants might not be able to change their maximum available capacity (other than for physical plant reasons) at certain times before dispatch. This option therefore further increases the ahead visibility and certainty to the operator regarding the resources that will be available in real-time.

OverviewThis design option follows a similar process as the system security ahead market (shown in Figure 7), but places mandatory ahead participation requirements in all markets and for all participants and has more stringent gate closure rules.

The requirement of receiving ahead energy schedule means that participants will have additional incentive to firm up their supply. However, it still leaves it to the participants to choose the mostcost-effective firming option for their requirements. For example, they could do this by eitherimproving the physical availability of their assets to reduce exposure to high real-time energy prices when supply demand condition is tight. Alternatively, they could be more conservative in their ahead bid and only bid in the “firm” component of their capacity. This would bring additional peaking resources into the merit order, which will be cleared in the ahead scheduling and receive an ahead energy award. These resources will then be incentivised to commit for supply in the real-time.

Gate closure rules regarding rebidding could also be considered in this option. While participants could still re-bid through moving quantities among different price band, gate closure could be applied to changing maximum available capacity. Participants might not be allowed to change maximum available capacity other than for physical reasons at certain times before real-time dispatch. This further reduces the risk of supply shortages before real-time, yet still allows participants the flexibility to meet their commercial requirements by moving quantities to higher price bands.

While more related to specific product design, this option could consider placing direct service obligations on individual participants for supplying certain resources. This could partially offset demand for some services. For example, requirement for individual renewable plant to hold certain headroom under certain market condition could reduce potentially volatility and lower the demand for operating reserve. This is another example for how this option can be designed to tighten the obligations in the ahead market, reducing the uncertainty that will be present in the real-time market, but potentially also reducing the flexibility to manage the changing conditions more efficiently.

Issues for considerationAs this option consists of similar design elements to the system security ahead markets, the issues raised in that option applies here as well. However, unlike the system security ahead market, the mandatory participation requirement in the current option requires all load, generation and storage resources to receive an ahead energy schedule. The commercial impact on participants needs to be investigated, especially for those that would not elect to receive an ahead energy schedule had participation been voluntary. The system benefit from additional firming needs to be weighed against the total cost incurred by individual participants, including additional equipment costs or financial impact due to mandatory participation. Similar consideration needs to be given if direct service obligations are placed on individual participants.

41

7. Conclusions and next steps

Current operations and identifiable trends as described in section 2 highlight that leaving the market design unchanged is unlikely to produce the most efficient results and may result in unacceptable risk to the secure and reliable operation of the system. The system is transitioning to one where the provision of energy no longer leads to the provision of system services without explicitly valuing, procuring and scheduling these services. Increasing complexity in the market with distributed resources, changing consumer behaviours, and changing technologies introduces additional uncertainty and variability. The market design needs to keep pace with this change and be able to accommodate more changes in the future. Without reforming the market design, regular out-of-market interventions and constraints will be required and the transition will continue to progress in a disorderly, uncoordinated, and costly way.

The ESB (through the AEMC and AEMO) will continue to develop the market design for new frameworks for system services and ahead market arrangements. Important issues to resolve to be able to properly assess the options include:

• How the new mechanisms will integrate with participant and system operator processes,including forecasting, adequacy assessments, bidding, information availability, physicalreality of the system, real-time operations, and settlement, for both energy and the systemservices.

• Designing detailed elements such as, for example, intervention pricing and compensationmethodologies to understand the incentives and penalties and relevant complianceframeworks required to ensure the success of the mechanism.

• Analysis of the appropriate timing of any new mechanism/s to account for the requiredcommitment of the resources to ensure the services will be available in real-time. This is atrade-off between the amount of time different resources need to enable their own availability (such as start-up times and organisation of their processes and staff) and more accurate information becoming available close to real-time. Commitment levels will also depend on the optimisation window (and whether a multi-interval optimisation process will be used) and the ability to adjust schedules intraday.

• Understanding the incentives and associated requirements for participation in andobligations arising from the ahead mechanism/s.

• Analysing the outcomes each option may produce under an array of plausible futurescenarios and therefore how it addresses the operational challenges that are arising andthe impacts on prices, including the forward energy contract market and impact on bills.

• Practical implementation pathways including relevant transitionary requirements tomanage the impact on operational processes and the contracting market, as well as thenecessary regulatory changes; and

• Interdependencies with other policy initiatives such as investment signals for reliability,locational pricing, innovation to benefit the consumer, integration of VRE, and integrationof DER and the demand side (including creation of two-sided markets).

A better understanding of how the options considered may integrate with the NEM will allow the ESB to carry out an evaluation of the options against agreed assessment criteria, including an economic analysis and alignment to the overall market design considering the other design elements (including investment frameworks and two-sided markets). An important component of the next phase will be to further develop the assessment criteria, leveraging the preliminary

42

discussion in the ESB’s Post 2025 Market Design Issues Paper published in 2019.33 This will include an assessment of the costs to implement compared to the benefits and trade-offs, especially compared to the counterfactual of retaining the current NEM design.

Any ahead mechanism option requires that the essential system services are valued. Determining the appropriate technical and market specifications for any new or revised system services will be a significant piece of work in and of itself. Developing an appropriate framework for how to best procure and schedule different types of system services is a crucial design step for the post-2025 market design work, as is ensuring that the regulatory arrangements are flexible and adaptive to allow the system services to keep up with the changing needs of the power system and technology innovation without unnecessary regulatory delays.

The ESB will recommend an ahead market design by the end of 2020 which works with other design elements, such as locational pricing, mechanisms to ensure investment in flexible, firm resources with the right technical capabilities, introduction of a two-sided market, and integration of DER. The recommended design will include a staged implementation, to make improvements as early as possible.

In the meantime, the ESB has identified some actions that, while they do not address all the issues and do not replace the need for comprehensive and enduring solutions, can improve visibility and commitment of resources, as well as improve the current processes to secure the system. These interim measures are outlined in a separate paper to the COAG Energy Council.

33 ESB (September 2019), Post 2025 Market Design Issues Paper,http://www.coagenergycouncil.gov.au/publications/post-2025-market-design-issues-paper-%E2%80%93- september-2019

43

A Abbreviations and Technical Terms

AEMC Australian Energy Market Commission

AEMO Australian Energy Market Operator

AER Australian Energy Regulator

COAG EC Council of Australian Governments Energy Council

dispatch Throughout this document, dispatch has a general meaning referring to the process of scheduling, committing, and providing targets in the electricity market.

DUCS Daily-UCS

ESB Energy Security Board

FCAS Frequency Control Ancillary Services

GPG Gas-powered generation

HUCS Hourly-UCS

ISP Integrated System Plan

NEM National electricity market

NEMDE National electricity market dispatch engine

NSCAS network support and control ancillary services

RERT Reliability emergency reserve trader

RIS Renewable Integration Study

STPASA Short Term Projected Assessment of System Adequacy

TNSP Transmission Network Service Provider

UCS Unit commitment for security

VRE Variable Renewable Energy

44

45

Contact details:Energy Security BoardE: info@esb.org.auW: http://www.coagenergycouncil.gov.au/energy-security-board