Survey of Multi-Agent Systems for Microgrid Control

Abhilash Kantamnenia,∗, Laura E. Browna, Gordon Parkerb, Wayne W. Weaverc

aDepartment of Computer Science, Michigan Technological University, United StatesbDepartment of Mechanical Engineering - Engineering Mechanics, Michigan Technological University, United States

cDepartment of Electrical and Computer Engineering, Michigan Technological University, United States

Abstract

Multi-Agent systems (MAS) consist of multiple intelligent agents that interact to solve problemsthat may be beyond the capabilities of a single agent or system. For many years, conceptualMAS designs and architectures have been proposed for applications in power systems and powerengineering. With the increasing use and modeling of distributed energy resources for microgridapplications, MAS are well suited to manage the size and complexity of these energy systems.The purpose of this paper is to survey applications of MAS in the control and operation of mi-crogrids. The paper will review MAS concepts, architectures, develop platforms and processes,provide example applications, and discuss limitations.

1. Introduction

The successful operation of a power system depends largely on its ability to economicallyand reliably meet load demands of residential, commercial and industrial customers. Early powerutilities employed human dispatch operators equipped with Supervisory Control and Data Ac-quisition (SCADA) systems to manage plant control, protective relaying, transmission switchingand communication protocols, along with economic operation of large interconnected powerplants. While SCADA systems offer timely and detailed monitoring of traditional grid resources,the raw data generated often contains only implicit information. Additional analysis by engineersusing multiple data sources is often required to obtain explicit information about power systemoperations. Such manual analysis of data can be time-consuming. For example, in the event ofgrid failure SCADA systems can generate thousands of fault records in a matter of only a fewhours, real-time manual analysis of which is unfeasible [1].

Autonomous control of power system operations using Multi-Agent Systems (MAS) has beenshown to overcome many such limitations [2]. MAS are composed of multiple intelligent agentsthat interact to solve problems that may be beyond the capabilities of each individual agent [3]. Inrecent years, MAS have been employed in a wide range of power system applications includingmodeling of electricity markets[4], grid protection [5], fault restoration [6] and grid control [7].In 2007, a comprehensive review of MAS for power engineering applications was conductedby the IEEE Power Engineering MAS working group regarding the technologies, standards and

∗Corresponding Author: Abhilash Kantamneni; Tel, 906-487-4302; Fax, 906-487-2283Email address: akantamn@mtu.edu (Abhilash Kantamneni)URL: www.cs.mtu.edu/~akantamn/ (Abhilash Kantamneni)

Preprint submitted to Engineering Applications of Artificial Intelligence July 17, 2015

tools for building MAS [8] and concepts, approaches and technical challenges within the field ofMAS that are appropriate to power engineering applications [9].

Recently however, technological advancements, security concerns, regulatory policy and en-vironmental considerations are changing the landscape of electricity generation and transmissionby reducing the grid’s reliance on large centralized generation facilities. Significant changes toderegulation and competition in the electrical industry over the past two decades led to the emer-gence of wholesale energy markets reliant on the decentralized decisions of generation firmsin contrast to utility based centralized generation units [10]. Consumer demand for clean en-ergy and government regulation is driving the increasing proliferation of Distributed Energy Re-sources (DERs) like photovoltaics (PV), fuel cells, and wind power into the modern electric grid.After the Northeast blackout of 2003, smaller scale localized generation systems emerged as akey contender in supplementing the existing power system to address the inability of a traditionalpower systems to provide for growing use of electricity [11]. More recently after Hurrican Sandy,several governmental agencies recommended investment in resilent energy resources [12].

Microgrids have emerged as an effective paradigm to manage DERs. A microgrid is anintegrated energy system consisting of interconnected loads and distributed energy resourcesthat operates in parallel with the primary power grid, or in a standalone “islanded” mode [13].In the event of a failure, the generation and corresponding loads of the microgrid can be isolatedfrom the distribution system without harming the integrity of the transmission infrastructure.Microgrids help facilitate rapid integration of DERs, offering “plug and play” capabilities withoutrequiring the re-engineering of the distribution system control architecture [14]. Microgridsare seen as a future power system configuration providing clear economic and environmentalbenefits. Extensive efforts are in progress across the world to demonstrate microgrid operatingconcepts in laboratories and in pilot installations [15]. In America alone, the Department ofEnergy is expected to oversee the development of commercial scale microgrid systems capableof reducing outage time of required loads by over 98% at a cost comparable to non-integratedbaseline solutions while reducing emissions by at least 20% and improving energy efficiencies bymore than 20% [16]. Reviews of microgrid concepts, operations and applications are availablein [14, 17, 18, 19, 20, 21, 22]

The purpose of this paper is to survey applications of MAS in the control and operation of mi-crogrids. Section 2 reviews agent and MAS concepts. Section 3 discusses multiagent interactionand coordination through MAS architectures, Section 4 describes a conceptual process, softwaretools and platforms (JADE, ZEUS and VOLTTRON) available for developing MAS specificallyfor microgrid control. Section 5 surveys the literature for demonstrative examples of MAS as analternative to traditional control of microgrids for applications including market operations, faultlocation and service restoration. Some limitations of agents and MAS are discussed in Section 6.

2. MAS Concepts

An agent is a computer system that is situated in some environment, and that is capableof autonomous action in this environment in order to meet its design objectives [23]. Agentstake sensory input from their environment, produce output actions that affect it. Agents can bedescribed with several properties:

• Autonomous: Agents exert partial control of their actions and internal state, seeking toinfluence outcomes without the intervention of humans or external devices.

2

Agent

Environment

SensorInput

ActionOutput

Figure 1: A simple agent in its environment [23]

• Social: Agents can communicate with humans, external devices or other agents to coordi-nate actions and satisfy their objectives.• Reactive: Agents react in a timely fashion to changes in their environment.• Proactive: Agents exhibit goal-oriented behaviors and take initiative to satisfy objectives.

Simple reflexive agents react to their environment without cognition of past or anticipationof the future state of the environment. Their reasoning is based on explicit knowledge or modelsof the environment. Learning agents have the ability to make improvements to their performanceelement by seeking feedback on their actions and interactions with the environment [24].

Sycara et al. [25] characterized Multi-Agent Systems (MAS) as distributed systems with twoor more agents, where:

• Each agent has incomplete information for problem solving. Agents can achieve globalobjectives through competition, collaboration or other interactions.• There exists no system for global control. Individual agents can cooperate with other

agents in the MAS to achieve individual objectives, or coordinate to maximize globalutility.• Data and environment are decentralized, all agents can affect changes in the environment

within their own ’spheres of influence’.• Computation is asynchronous, agents can carry their tasks independently without having

to wait for a central control signal.

Stone and Veloso [26] present many example implementations of MAS (Figure 2). Agents candiffer (heterogeneous) or be identical (homogeneous) in goals, actions and domain knowledge.Homogeneous agents differ in sensor inputs and action outputs, hence do not behave identically.Depending on the inputs they receive, each homogeneous agent can make an independent deci-sion regarding their action response. Agents can communicate with each other through a centraldirectory, or can transmit information directly between each other. Non-communicative agentscan still affect each other indirectly, either actively through sensor inputs or passively by alteringthe environment.

MAS allow for the distributed control of microgrids as an alternative to traditional hardwarebased centralized control. In some domains, the inherent complexity of a system makes it dif-ficult to obtain a priori knowledge about potential links between sub-problems. Consider forexample, the problem of optimizing the market performance of a grid connected microgrid with

3

Agent #1

• Goals• Actions• Domain

Knowledge

Environment

Agent #2

• Goals• Actions• Domain

Knowledge

Agent #3

• Goals• Actions• Domain

Knowledge

Agent #4

• Goals• Actions• Domain

Knowledge

Figure 2: MAS with homogeneous (#3 and #4), heterogeneous, communicating (#2, #3 and #4), and non-communicating(#1) agents [26].

multiple Consumption Units (CU) and Production Units (PU) [27] (Figure 2). Each CU or PUmight have different owners and therefore different objectives. In open market auctions, PUswill try to maximize their profits while CUs will try to minimize the cost of purchasing power.Additionally, the system has a global objective of load balancing at the end of each auction pe-riod. Traditional approaches to solving this optimization problem requires the enumeration of allacceptable Selling Prices (SP) and Buying Prices (BP) for each individual unit connected to themicrogrid. As the number of units in the microgrid increases, solving the problem in real-timebecomes difficult.

MAS present an effective way of decomposing complex problems into a number of simplersub-problems. By modeling each unit as an autonomous agent, each agent attempts to achieveits individual objective of maximizing its utility. When the auction period begins, CU agentsactively negotiate with PU agents to meet their load demands. If the initial price bid by theCU agent is rejected by the PU agent, the CU agents make progressively higher bids till theirdemand price is met by a PU, failing which they autonomously choose to buy energy directlyfrom the grid. The global objective of meeting load demand therefore emerges out of inter-agentcollaboration and coordination in real-time. Even in the absence of a priori knowledge of thecomplete system, agents can employ machine learning techniques to learn how to solve problemsin a stochastic environment without a central controller [28].

A centralized hardware-based control system may need to be redesigned to accommodatestructural changes to a microgrid. Agents, however, are independent of their environment. Inthe earlier example of microgrid market operations [27], all CU agents are homogeneous, asare PU agents. The same set of agents programmed for one instance can be deployed for mul-tiple instances without the need for redevelopment. When built on open architectures, agents

4

Microgrid

ConsumptionUnit

ProductionUnit

MainGrid

DemandPrice

ProductionPrice

Buying PriceSelling Price

Figure 3: Market operations of a microgrid using MAS, adapted from [27].

are programming language and platform independent. Such programmable MAS are easily dis-tributable and offer “plug and play” capabilities, allowing flexibility for future expansion. Con-sider the problem of service restoration in a microgrid, after a section is isolated due to a fault. Under fault conditions, circuit breakers in a grid system are designed to trip rapidly to isolateand localize the fault. A centralized controller might lack the ability to adapt quickly to suchsudden changes in network structure. Xu et al, [29] implemented a distributed MAS controlsystem, where each bus of the microgrid is associated with a node agent (NA). Each NA onlyhas access to local generation and load information. An independent infrastructure facilitatesinter-agent communication, even in the absence of a distribution line. In the event a fault, indi-vidual NAs use the communication infrastructure for global information discovery through localcommunications with neighboring agents. A composite picture of the entire system is arrived atthrough interactions of multiple agents with local information. Service to loads in the microgridcan subsequently be restored after all the global information has been discovered. Simulationstudies demonstrate the effectiveness of this approach as an alternative to traditional computingalgorithms.

A centralized approach to solving sub-problems in a complex system might have unpre-dictable outcomes due to the impossible task of modeling all possible interactions and subsequentperturbations. The abstraction of sub-problems into autonomous agents also allows reflexiveagents to achieve their design objectives in an asynchronous manner, without waiting for exter-nal commands to carry out their tasks. Such a decentralized approach helps faster operation anddecision-making process, leading to effective microgrid control in a time-sensitive manner [22].MAS have increased fault tolerance and resiliency owing to redundancy built into the system [8].The roles of each agent can be masked or duplicated into alternative backup agents that can becalled upon to execute objectives in case of agent failure.

5

Table 1: Summary of MAS architectures for microgrid control.Architecture

[Ref. Implementation] Agent Type Role

Centralized [30] Cognitive High level decisions, communicationReactive Quick response

Distributed [31] Local Local information discovery, communicationsTwo-levelHierarchical [32]

High-level Infrastructure management, inter-microgrid com-munication, low-level agent scheduling

Low-level Accept schedules, asset management

Three-LevelHierarchical [33]

High-level Critical decisions, data and policy managementMid-level Fault location, grid-connected/islanded mode

switchingHigh-level Sensor management, hardware I/O

3. MAS Architecture

MAS may consist of large numbers of agents operating in rapidly evolving dynamic environ-ments. Since data and environment are decentralized, the roles and responsibilities of intelligentagents need to be clearly defined to resolve potential conflicts that may arise through agent in-teractions. While a conceptual MAS design process is discussed in a later Section 4, a basicstructure of a MAS can be broadly classified into the following architectures (see Table 1).

3.1. Centralized

A centralized architecture is characterized as a collection of simple homogeneous, non-communicative agents that are managed by a single control center in a master-slave relation-ship. Such an arrangement closely reflects traditional zone based control strategies with someadditional functionality.

Dimeas and Hatziargyriou [30] describe two classes of agents: reactive and cognitive. Cogni-tive agents are equipped with increased intelligence and higher-level communication capabilities.The central controller as a cognitive agent communicates with reactive agents and manages theoperation of non-critical loads (see Figure 4). In the event of a power shortage, the central con-troller disconnects non-important loads. Additionally, some control is also distributed throughreactive agents that lack higher-level decision making capabilities. Such agents are used in ap-plications that demand a quick response. In the event of a fault, an under frequency relay withonly access to local information can trip a circuit autonomously without coordinating its actionswith a central controller. Other implementations of centralized architecture for MAS control ofmicrogrids are available at [27, 34].

3.2. Distributed

A distributed architecture is characterized as a collection of communicative agents managedby a single layer control structure. Each local agent is responsible for and has knowledge aboutits own part of the network, indeed no single agent has complete knowledge of the whole domain.

Instead, individual agents are allowed to discover global information through communicationand coordination with their neighbors. A single common communication framework facilitatesinteraction among all agents. In [31], the authors propose a distributed system of self-organizing

6

CognitiveAgent

ReactiveAgent

ReactiveAgent

ReactiveAgent Reactive

Agent

ReactiveAgent

ReactiveAgentReactive

Agent

ReactiveAgent

Figure 4: Generic schematic of a centralized architecture for MAS microgrid control.

Energy Source, Energy Storage and Load agents, Individual agents can register and publishtheir capabilities to a directory, from which other agents can evaluate and request services (seeFigure 5). A Load Agent can look up all available Energy Source agents and solicit bids forenergy supply. Energy agents can communicate with Energy Storage agents to schedule storageevents. The distributed architecture can also help build robust systems with built-in redundancy,with agents being able to re-organize and cope with the loss of other agents.

Some centralized and distributed MAS architectures may not be able to fully exploit col-laboration, competition, consensus building or other advanced functionalities offered by MAS.For example, a Load Agent in [31] can only select from available bids and cannot simultane-ously negotiate for competitive prices from many Energy Source agents. Simple communicationprotocols only permit one-on-one interactions between individual agents. The absence of suchfunctionality might result in sub-optimal allocation and management of critical resources.

3.3. HierarchicalHierarchy is characterized by some agents having authority over the actions of other agents [35].

Most MAS implementation of microgrids in literature employ a three-level hierarchal archi-tecture. Typically, the upper level agents are responsible for critical decisions, handling largeamounts of data and maintaining overall policy, communication schedules and protocols. Mid-dle level agents make decisions about switching between grid connected and islanded modesof microgrid operation, minimize losses, direct the actions of sub-agents for fault location andservice restoration. The lower level agents interact with actual sensors and devices that are con-nected to the microgrid. They sense and control components or devices of the microgrid, such asgrid breakers, distributed energy, energy storage devices and controllable loads. Such a hierar-chy offers good scalability through clear delineation of roles to agents to ensure robust real-timeoperational control [36]. An example of three level architecture is implemented in [33] with self-interested agents managing power flow locally (see Figure 6). The top level Supervisory Agent

7

Communications& Directory

LocalAgent

LocalAgent

LocalAgent

LocalAgent

LocalAgent

Figure 5: Generic schematic of a distributed architecture for MAS microgrid control.

ensures a strategic supervision of brokerage between energy consumers and suppliers, buying orselling from the open market and disconnecting loads and cells in the event of a blackout. TheSources Agent is a collection of agents regulating power-generating sources in the system withsome crossover with the Loads Agent in the case of batteries. The Transportation Agent commu-nicates with the top layer to receive and distribute scheduling information, orders connection toor disconnection from the network and also reads real power data from the microgrid. A simu-lation of this system was shown to perform a dynamic reconfiguration of its structure to quicklyredirect energy flow as well as disconnecting loads to protect itself.

In [32], the authors propose a framework for MAS-based microgrid control based on a twolevel hierarchical architecture: the upper level MicroGrid Central Controller (MGCC) is respon-sible for providing basic infrastructure and services and the lower level microgrid Agent Platformwhich in turn is comprised of Micro Source Controller (MSC) and Load Controller (LC).

MGCC contains individual agents that are responsible for generation scheduling, marketbidding, shifting load connection requests, issuing curtailment actions and interfacing with otheragents in the system. The MSC tracks changes in active power output and sends power sellingbids to the MGCC. The LC registers shiftable and curtailable loads into the system, and is incharge of executing shifting and curtailment commands received from the MGCC. In [27], agentsin the MGCC operate in a collaborative mode to maximize the gain of selling energy to themain grid, and in a competitive mode to reduce their individual operating cost. The idea isextended in [7, 37, 38, 39, 40] to include market participation through Market Operators (MO)and for advanced “plug and play” capabilities of future microgrids, where the agent in the highestlevel is only involved in decision making process of the most complex tasks (see Figure 7).Scaling up to even larger systems, multiple microgrids are combined in [41] and the merits ofusing a tree hierarchy structure over having equal agents engaging in a reciprocity relationshipis discussed. Other implementations of hierarchal MAS architectures in literature are discussedin [40, 42, 43, 44, 45, 46, 47, 48].

8

GridPVWind Load Load

BrokerPolicy

ManagerDisconn.Manager

Node FeederNetworkSimulator

Load AgentSource Agent

Supervisor Agent

Transportation Agent

Figure 6: A three level hierarchal architecture for MAS microgrid control from [33].

4. Development Platforms

A complete description of a conceptual design process for creating MAS is beyond the scopeof this paper, but Ricordel and Demazeau [49] describe a generic four stage development process:

1. Analysis : Modeling agent roles and behaviors. Identifying the application domain andproblem.

2. Design : Defining solution architectures for problems identified in the Analysis step3. Development : Programming agent goals, ontology and functionality4. Deployment : Launching generated MAS, run-time agent management, message passing

and data processing.

Building a MAS requires an agent development environment that supports at least somestages of the MAS conceptual design process. Comprehensive reviews of several agent develop-ment platforms are available in [50, 49, 51, 52]. This section focuses on platforms most oftenused by researchers for developing MAS for microgrid control applications. Table 2 provides asummary of the same.

Foundation for Intelligent Physical Agents (FIPA) specifications [54] define rules for the ex-istence, operation and management of generic agents that can be combined to make complexsystems. FIPA also allows for the abstraction of the agent development platform from the lan-guage used to program the agent, its environment, domain, data and external devices. In order tofacilitate communication between agents on different platforms and networks, FIPA also spec-ifies a standard messaging language, Agent Communication Language (ACL), encoded in text.The FIPA reference model (Figure 8) prescribes the following agents as essential for agent de-velopment platforms:

9

MGCC

MO

LCLC LC

Microgrid 1 Microgrid 3

Microgrid 2

Figure 7: A two level hierarchical architecture for MAS microgrid control, adapted from [27]

• Agent Management System : Supervisory platform access, creation, deletion and managingagents on the platform.• Agent Communication Channel : Inter and Intra platform agent communication.• Directory Facilitator : Message board service through simple queries. Allows agents to

offer and discover services offered by all agents on the platform.

4.1. JADE

JADE (Java Agent Development Environment) is an open source agent development soft-ware framework [55] for building FIPA compliant MAS. JADE provides tools for building dis-tributable agents across multiple hosts while supporting parallel and concurrent agent activi-ties. JADE supports the Design and Deployment stages of the MAS conceptual design pro-cess, giving programmers the freedom to abstract agent design. JADE is fully implementedin Java programming language. Additionally, it comes with Graphical User Interface (GUI)tools for debugging and is freely available for download [56]. Tools for combining MatlabSimulink with the JADE environment are also freely available for research purposes [57]. Someexamples of MAS for microgrids developed using the JADE agent platform are available in[32, 37, 40, 43, 44, 58, 59, 60, 61, 62, 63, 64, 65]

4.2. ZEUS

ZEUS [66] is a FIPA compliant open source agent development platform implemented in theJava programming language. It provides users with a graphical user interface (GUI) and a run-time environment. Along with ACL support for agent communication under FIPA compliance,ZEUS also supports Knowledge Query and Manipulation Language (KQML) based communi-cation. ZEUS supports all stages of the MAS conceptual design process and is freely availableto download for research purposes [67].

ZEUS allows the modeling of agent roles using combination of class diagrams and prede-fined roles common in most management systems. This design environment restricts the need

10

AgentManagement

System

DirectoryFacilitator

AgentCommunication

Channel

Internal Platform Message Transport

Agent

Software

Figure 8: FIPA reference model of an agent platform, as described in [53].

for developing new formalism, thereby making MAS development more accessible to a wideraudience [49]. Additionally, ZEUS also provides a runtime environment, and various assis-tant tools for debugging, observing coordination strategies, general purpose planing and processscheduling.

Despite these advantages, new developers might face some challenges in creating new appli-cations using the ZEUS platform owing to weak documentation [52]. Some examples of MASdeveloped for microgrids using the ZEUS agent platform are available in [36, 68]

4.3. VOLTTRON

VOLTTRON [69] is a distributed agent execution framework designed by Pacific NorthwestNational Laboratory (PNNL) specifically for use in electrical power systems. The open sourceand modular platform is intended to support transactions between networked entities over thegrid. Communication is established through a central ‘MessageBus’ in the form of topics andsubtopics (for example, ‘topic/subtopic/subtopic/’ = ‘weather/location/Temperature/). The con-trol architecture is modeled as a three-level hierarchy of agent classes:

• Cloud Agent : Publishing data to and from a remote platform• Control Agent : Interact with devices• Passive Agent : Interact with sensors and record data

A combination of agent classes can be employed to derive a variety of agents, and the VOLT-TRON development page on Github provides clear examples [70]. While a prototype implemen-tation of VOLTTRON is available in Python, the platform is programming-language agnostic.VOLTTRON also provides driver support for most Modbus [71] and BACnet [72] devices.

11

Table 2: Summary of key differences between JADE, ZEUS and VOLTTRON MAS development platformsJADE ZEUS VOLTTRON

Free and Open Source Yes Yes YesFIPA Compliant Yes Yes No

Editor Command line GUI Command linePlafrom Support Active. Updated Dec, 2014 Discontinued Active. Updated April, 2015

Programming Language Java based Java based Programming language independentAdvantages Stable platform Ease of development Hardware driver support

Disadvantages Challenging for new developers Weak documentation Limited industry adoptionIdeal Application Scalable Microgrids Rapid Prototyping Building Energy Management

4.4. Discussion

Several other MAS platforms exist but were not included for comparison because they havenot been actively updated recently, e.g., Aglets [73], or not used within the power domain, e.g.,SkeletonAgent [74].

The MAS described in a majority of papers surveyed here were built using the JADE plat-form. Studies evaluating various MAS development toolkits [50, 75] have shown that JADE canbe used to build robust, complex and scalable MAS for most applications.

Some reviews [49, 76] of agent development platforms suggest that the GUI of ZEUS, alongwith tools like report generation, statistical analysis and debugging make it a user friendly devel-opment environment for rapidly building scalable and distributable MAS, especially for begin-ners unfamiliar with the agent design paradigm. The same reviews also suggest that the ease ofuse may come at the cost of lost functionality. Experienced programmers might find it easier tocustomize or debug the Java language of JADE agent code over fine-tuning the GUI controls ofZEUS.

As a relatively new and continually evolving platform, adoption of VOLTTRON for MAScontrol of microgrids in research and industrial applications is currently limited, compared towidespread use JADE or ZEUS. However, the open source development platform, broad driversupport and programming language independence allows developers to prototype and demon-strate MAS control of grid devices even without access to testing hardware. VOLTTRON willtransition from a Department of Energy supported project to the Transactional Energy Consor-tium for continued development (e.g., integration with GridLAB-D [77]).

In summary, JADE is best suited for advanced developers building stable and scalable MAScontrol for microgrids. ZEUS could be best employed by new developers for rapid prototypingand testing of MAS concepts for microgrid control. VOLTTRON is best employed by facilitiesand building managers for managing sensor and instrumentation data.

5. Applications of MAS in microgrids

The Consortium for Electric Reliability Technology Solutions (CERTS) identifies some keyissues in the control of microgrids [78]:

• New microsources can be added to the system without modification of existing equipment.• Microgrids can connect to or isolate themselves from the grid in a rapid and seamless

fashion.• Reactive and active power can be independently controlled, voltage sag and system imbal-

ances can be corrected.

12

GA2GA1 GA3

LA1 LA2 LA3

Figure 9: Auction based mechanism using simple bilateral contracts between Generator (GA) and Load Agents (LA),adapted from [82].

• Microgrids can meet the larger grid’s load dynamics requirements.

Traditional centralized control infrastructures for microgrids with many devices may incur largecosts in communication infrastructure and complexity of the centralized control supervisor. Al-ternatively, an autonomous local controller needs only to take action to local events using localinformation. This may lead to more customized local control systems for uniquely different util-ities in the system giving greater control over real time operation of a microgrid [22]. Indeed,“local information plus communication produces global control” [79]. The characteristics ofMAS discussed in Section 2 lend themselves very well to microgrid control application in thecontext of Market Operation, Fault Location and Service Restoration.

5.1. Market OperationsSignificant changes to deregulation and competition in the electrical industry over the past

two decades has led to the emergence of wholesale energy markets reliant on the decentralizeddecisions of generation firms in contrast to utility based centralized generation units [10]. Aselectricity markets continue to evolve, fully functioning markets are distinguished by the pres-ence of a large number of utility companies, power brokers, load aggregators and marketers thatare in direct competition. Market forces drive individual participants to develop their own uniquebusiness strategy, risk preference and decision models. Such detailed and complex economic sys-tems cannot be adequately analyzed by conventional models that utilize a single decision makerand a global objective for the entire system [80]. The wholesale market is a dynamic networkof fuel, resource forecast, bilateral and auxiliary markets. The limitations of traditional model-ing methods in dealing with the added complexity of real time demand and supply optimization,transmission limits and unit commitment constraints has been shown to be overcome by agentbased simulation models [81].

Agents facilitate distinct players in real-time markets to optimize their performance by max-imizing their individual utility through an auction-based mechanism. A typical strategy is toestablish bilateral contracts between buyers and sellers in the market. During the negotiationperiod, a buyer sends out a request with price expectations to all the sellers in the market, and thesellers respond with a price based on availability, system and transmission costs. In [82], Genera-tor Agents (GA) interact with their affiliated Distributed Energy Resource (DER) determine unitoperation, fuel and maintenance cost along with preferred markup. Load Agents (LA) determinethe maximum unit price it is willing to pay for a quantity of power required during a specificperiod of time. GA try to maximize their profits by selling energy above the production costs

13

MAG

GAG1GAG2

GAG3LAG3

LAG2LAG1

Figure 10: Auction based mechanism using a market operator with Microgrid control Agent (MAG), Generator Agents(GAGs), and Load Agents (LAGs),adapted from [84]

while LA try to minimize costs by purchasing energy below unit price. The proposed methodimplements simple bilateral contracts between LA and GA that guarantees energy allocationsthrough competitive bidding (see Figure 9).

Another strategy is to establish a market operator that sends out a request for bids to allagents in the system at the start of each negotiating period. The market operator subsequentlyprocesses the bids, establishes market price and matches the buyers with the sellers [83]. A MASsystem that maximizes revenue of a microgrid in the power markets is discussed by Funbashi etal, [84]. The proposed method consists of several Loads Agents (LAGs), Generator Agents(GAGs) and a single microgrid Control Agent (MAG) implemented in a three-level hierarchicalarchitecture. The GAGs participate in the supply side regulating the operation of the generators,the LAGs operate on the demand side creating agents for buying power from the microgrid.MAG is responsible for optimizing the operation of the microgrid using a negotiation algorithmthat selects Seller/Buyer pairs based on the lowest internal selling price and the highest internalbuying price respectively (see Figure 10). When a conflict arises, the biggest generation reserveand the heaviest load demands are selected as a Seller/Buyer pair respectively. A large numberof simulations over varying testing conditions were shown to have promising results. A moredetailed description of a similar architecture including the software implementation is discussedin [27]. Preliminary results from the testing of the MAS on a laboratory microgrid demonstratedthe feasibility of the approach.

In addition to maximizing the utility of individual players, microgrids operating in real-timepower markets may need to satisfy additional objectives: emission and noise levels, operationalcosts, electricity prices, intermittent and non-dispatchable renewable energy integration, storagescheduling etc. Decentralized control using MAS may offer flexibility in solving such multi-obective optimization problems. Individual agents with local information acting in self-interestcan also co-operate to achieve mutually beneficial goals. Two scenarios investigated in [85]using a test Microgrid with DERs, storage, diesel generator and loads illustrate the benefits ofsuch an approach. Disturbances are introduced in the stable operation of a microgrid in theform of a rapid change in spot market prices, and sudden loss of solar resource. The microgridwas shown to be able to meet load demand through the co-operation of the generation, storage

14

and PV agents. In addition to changing operational scenarios, collaborative decision making byindividual agents was also shown to adapt to changing user defined goals and objectives [86].

Market-driven management of distributed, micro storage devices is considered by Vytelingumat al. [87]. The authors cast the problem as a multi-player game showing that the Nash equilib-rium is equivalent to the minimizing the total electricity generation cost achieved by reducingexpensive peak demand power. Faqiry et al., [88], use a genetic algorithm approach to investi-gate energy trading patterns for a microgrid using renewable generation. The ability to scheduleloads is exploited in conjunction with a an optimization cost function that monetizes consumersatisfaction. Trinklein at al., [89], consider the value stream associated with networked micro-grids. They illustrate that a mutually beneficial solution between microgrid owners and utilitiesmay be achieved. This is accomplished by managing the network’s aggregated load such that itdoes not add to peak demand during nominal operation and could provide regulation reserve.

With the advent of increasingly efficient consumer storage devices, demand side units canbe modeled as proactive agents participating in open electricity markets with the intent to lowerpeak demand thereby reducing the need for expensive peaking generators, lowering both carbonemissions and consumer energy costs [87]. MAS have been shown to handle network constraintsthat exist in real world electricity markets, by acting in collaboration with ’smart’ meters in-stalled at the consumer end [90]. Regulatory agencies appointed to manage competitive electricmarkets, preserve security of supply, enhance privatization of supply and ensure overall systemefficiency have used agent based simulations to analyze market design policy issues for electricitymarkets [91].

A comprehensive review of electricity market modeling literature [92] shows that the conceptof agent based simulation as a test bed for the electricity sector can provide additional insights formarket and policy design. Other recent implementations of MAS for microgrid market operationsare given in [40, 60, 65, 93, 94, 95, 96, 97, 98, 99, 100]

5.2. Microgrid Protection

Microgrids consist of a distribution network that connects several generation units includingPV, wind, fuel cells, combined heat and power (CHP) units, microturbines and distributed storageunits that improve the performance of a microgrid by damping peak surges in electricity demand,countering momentary power disturbances, providing outage ride-through while backup genera-tors respond, and reserving energy for future demand [101]. The radial distribution network witha central power source undergoes a topological change into a new complex architecture whendistributed generation (DG) is added. With a large short circuit capacity, a more complex faultcurrent path, variation in dynamic response of the network and the unpredictable output powercharacteristics of dynamic DG units might render traditional over-current protection schemesineffective [102].

Protection must be extended to the microgrid in its two modes of operation, when connectedto the distribution network and in islanded mode. When a fault is discovered in the upstream net-work, the microgrid is disconnected from the distribution network, and must control its voltageand frequency, provide instantaneous real power difference between generation and loads, pro-vide the difference between generated reactive power and the actual reactive power consumedby the load to protect the internal microgrid. In the islanded mode, frequency control of unitslike micro-turbines and fuels cells is a challenging problem due to their slow response to controlsignals. The frequency control strategy should exploit in a cooperative way the capabilities of themicro sources to change their active power, load shedding or activating storage systems [103].

15

RA

ZAZA ZA

SASA SA

Figure 11: A three level MAS hierarchy for fault discovery and location in a MAS with Recloser (RA), Zonal (ZA), andSwitch Agents (SA), as implemented in [104].

Several MAS approaches have been proposed to detect faults in a real-time operation of a distri-bution network and subsequent disconnection and isolation of microgrids from the network.

Alwala et al, [104] implement a three level hierarchal architecture (Section 3.3) for MAS withtwo classes of agents: the upper level Recloser Agents (RA), a lower level Zonal Agent (ZA) andSwitch Agent (SA) (see Figure 11). The two classes of agents can coordinate to locate and isolatefaults in a microgrid based on sequence current magnitudes and current direction reversal duringa fault. RA monitors the status of reclosers at each substation. ZA has local information suchas power flow and sequence current magnitude and phase angle of all the distribution lines intheir respective. Each ZA has a single SA associated with that zone. When a permanent faultoccurs in the system, RA issues a lockout signal to the lower level ZA which then initiates afault location algorithm through its associated SA. Each SA checks its zone for sequence currentreversal or residual current magnitude in all lines that are connected to the violating line. Analternate method for identifying faults is also presented where nodes with no two-way powerflows are identified as faulty. Thus, while fault identification occurs at the upper RA level, faultlocation happens at the lower ZA and SA level.

Pipattanasomporn et al. [105] implement a two level hierarchy MAS to disconnect and sta-bilize the microgrid from the local utility when upstream outages are detected (see Figure 12).The authors described system with four classes of agents, namely a Control Agent, a DG agent,a User Agent and a Database Agent. In grid connected mode, the control agent in communi-cation with the user agent and the DG agent determine the amount of loads to be shed and theamount of power to be produced internally in order to stabilize the microgrid. All agent actions– from detecting the fault, disconnecting the main circuit breaker, disconnecting the non-criticalloads to stabilizing the grid require half an electrical cycle. The control agent also drives themicrogrid into the islanded mode by disconnecting the main circuit breaker. In islanded mode,

16

DatabaseAgent

ControlAgent

DERAgent

UserAgent

Figure 12: A two level MAS hierarchy for fault discovery and location in a MAS, as implemented in [105].

the user agent and the DER agent balance the demand and supply by controlling the voltage andfrequency at prescribed limits.

In [106], generic agents manage relays and communicate through standard Ethernet protocolswhich are independent of network topology, thereby providing a scalable and dynamic approachfor protecting distributed networks. In the event of a critical failure in a distribution line, suchan independent communication infrastructure allows agents to continue interaction with otheragents on the distributed MAS. The generic agents can also be copied into different componentssuch as switches, breakers and other protection devices, building robust systems with built inredundancy that are resilient to single point fault failure. Examples of MAS for microgrid pro-tection applications are also available in [64, 107, 108, 109, 110, 111, 112]

5.3. Service Restoration

After a fault has been located and cleared, it is essential to restore a system to its opera-tional efficiency. Several approaches to using MAS in distributed power systems are proposedin the literature [6, 113, 114, 115, 116] indicating that for fault detection, isolation and servicerestoration, MAS could provide an alternative to a centralized processing system.

MAS are particularly more useful in microgrid applications owing to the complexity of thelarge networks of distributed energy resources, loads and storage units that preclude centralizedcontrol. In [117], the authors describe a MAS architecture that strikes a balance between theintra-microgrid objectives defined by local operator and the situational demands of the microgridcollective. Under normal operating conditions, agents operate under self interest by maintaining

17

power to the local vital loads at all times and will seek to export any excess power to other mi-crogrids through communication with other local agents. In an emergency condition, after a faultis located and isolated, the agents begin the restorative phase by shedding non-vital loads andtransmit surplus requests seeking additional power. Agents are capable of negotiating temporarypower contracts during which the microgrids with excess generation pair with those that haveshed load during the emergency. The MAS transitions to a normal operating condition undernew network topology, while periodically checking to see if the emergency condition has beenrectified.

In [118], the authors discuss a three layer MAS architecture that can overcome the static con-straints (active power balance, reactive power abundance and power flow) on maximizing micro-grid fault restoration. When a fault occurs, the lower-level Load Agents (LA) and local agents(SWA) communicate with a Distributed Generation Agent (DGA), which calculates availabletransfer capacity and requests additional support from the middle level Microgrid Agent (MGA).MGA communication protocols are overseen by the higher level Central Control Agent (CA).Multiple MGAs then coordinate efforts into optimizing load restoration by communicating theirbranch maximum transmission capacity and prioritizing the removal of the most non-criticalloads designed to restore proper equilibrium to the microgrid with least load loss.

In [29], the authors propose a novel MAS where each agent makes synchronized load restora-tion decisions according to global information discovery through local communication. Eachnode agent only has knowledge of local generation and load information but no access to globalinformation. During the information discovery process, agents only communicate with theirdirect neighbors, and the global information is discovered based on the Average-Consensus The-orem. Each agent is initialized with a matrix whose elements indicate the connection status ofvarious generators and loads. Through application of average-distance algorithm to each cor-responding elements of all information matrices, required information for restoration can beobtained. Theoretically, the proposed load restoration algorithm can be applied to systems ofany size and structure. Authors in [119] propose using wireless networks for communication andinformation discovery in microgrids for consensus based service restoration. Wireless commu-nication between agents was shown to facilitate microgrid monitoring and service restoration ata high flexibility and low cost. Other applications of MAS for microgrid service restoration areavailable in [63, 120, 121]

6. Benefits and Limitations of MAS

Several of the advantages of MAS for power system applications are again emphasized.These advantages pay particular attention to the expansion of Distributed Energy Resources andthe incorporation of renewable sources.

• Distributed Architecture : The nature of the distributed generation fits into the MAS archi-tecture schemes that rely on local information and decision making.• Flexibility : MAS enable flexibility in several ways: “plug and play” capabilities to change

the system and heterogeneous types of agents modeling heterogenous sources and loads.• Resilency : MAS can quickly respond and adjust to faults. Additionally, changes in net-

work topology (a load or generator being disconnected) will not interrupt both local andglobal system objectives (e.g., stability and efficiency).

Wooldridge et al. [122] identified general pitfalls in developing agent oriented processes.Challenges in using MAS for power system applications were reviewed in [8]. As with any

18

emerging research paradigm, MAS control of microgrids faces some limitations that impedewidespread adoption. Each of these limitations are also opportunities for future research direc-tions.

• Emergent behavior : The autonomous and distributed nature of agents might lead to un-predictable outcomes. While the goals and objectives of agents can be programmed, theeffect of run-time interactions cannot always be pre-determined [123]. This sort of emer-gent behavior might be beneficial under some circumstances (e.g., market operations), butinherent uncertainty may be a drawback in some applications (e.g., service restoration).• Portability : Hardware implementation of conceptual MAS designs and architectures can

be challenging. Most current implementations of MAS control of microgrids are softwaresimulations, e.g., virtual MATLAB Simulink testbed, and the performance of many MASapproaches on actual microgrid hardware is yet to be widely tested.• Scalability : Greater computational power available these days allows researchers to model

larger microgrids with many agents coordinating actions on a single platform. However,the ability of MAS to scale [124] with increases in problem dimensions (agents acrossmultiple platforms) or diversity (agents of multiple types) is not well understood.• Security : The shift from largely physical infrastructure towards smarter technology in-

creases the risk of security and privacy violations from malicious external actors and dis-ruptive elements.

7. Conclusions and Future Trends

This paper introduced the theory and concepts that make Multi-Agent systems (MAS) wellsuited for the operation and control of microgrids. Agent interaction, coordination and coop-eration was discussed in the context of MAS design architectures. A step-by-step conceptualframework and platforms for building MAS were introduced. The application of MAS in mi-crogrids for market operations, fault identification, fault location and service restoration wasreviewed with demonstrative examples from literature.

Microgrids are expected to be an integral part of the electricity grid of the future offeringimproved resiliency, integration of DERs, bi-directional vehicle charging, advanced storage anddemand management [125].

As greater computational power becomes more available, researchers are able to model in-creasingly complex interconnected microgrids. In simulations, MAS were shown to scale tomicrogrids with thousands of interconnected generators, loads, buses, breakers, reclosers andother grid elements [104]. Improvement in forecast models will lead to better prediction of loaddemand in disaggregated environments over shorter time horizons [126]. Agent communicationscontinue to evolve by adapting to changing grid communication protocols leading to improvedagent consensus building, faster response time and adaptability [127]. Standardization of MASarchitecture and practices will lead to greater system interoperability in the microgrid and smartgrid environment.

The inherent uncertainty of software complexity, hardware incompatibility and security riskto malicious external actors limit widespread adoption of MAS for the control of microgrids. Atrue test of MAS applications for microgrid control can only come from rigorous field tests ofhardware prototypes from multiple developers and vendors.

19

Acknowledgment

Research was sponsored by the Army Research Laboratory and was accomplished under Co-operative Agreement Number W911NF-13-2-0024. The views and conclusions contained in thisdocument are those of the authors and should not be interpreted as representing the official poli-cies, either express or implied, of the Army Research Laboratory or the U.S. Government. TheU.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation therein.

[1] Euan M Davidson, Stephen DJ McArthur, James R McDonald, Tom Cumming, and Ian Watt. Applying multi-agent system technology in practice: automated management and analysis of scada and digital fault recorder data.Power Systems, IEEE Transactions on, 21(2):559–567, 2006.

[2] Robin Roche, Benjamin Blunier, Abdellatif Miraoui, Vincent Hilaire, and Abder Koukam. Multi-agent systemsfor grid energy management: A short review. In IECON 2010-36th Annual Conference on IEEE IndustrialElectronics Society, pages 3341–3346. IEEE, 2010.

[3] Gerhard Weiss. Multiagent systems: a modern approach to distributed artificial intelligence. The MIT press,1999.

[4] Anke Weidlich and Daniel Veit. A critical survey of agent-based wholesale electricity market models. EnergyEconomics, 30(4):1728–1759, 2008.

[5] Qingle Pang, Houlei Gao, and Xiang Minjiang. Multi-agent based fault location algorithm for smart distributiongrid. In Developments in Power System Protection (DPSP 2010). Managing the Change, 10th IET InternationalConference on, pages 1–5. IET, 2010.

[6] Takeshi Nagata and Hiroshi Sasaki. A multi-agent approach to power system restoration. Power Systems, IEEETransactions on, 17(2):457–462, 2002.

[7] AL Dimeas and ND Hatziargyriou. Agent based control of virtual power plants. In Intelligent Systems Applica-tions to Power Systems, 2007. ISAP 2007. International Conference on, pages 1–6. IEEE, 2007.

[8] Stephen DJ McArthur, Euan M Davidson, Victoria M Catterson, Aris L Dimeas, Nikos D Hatziargyriou, Ferdi-nanda Ponci, and Toshihisa Funabashi. Multi-agent systems for power engineering applications part i: concepts,approaches, and technical challenges. Power Systems, IEEE Transactions on, 22(4):1743–1752, 2007.

[9] Stephen DJ McArthur, Euan M Davidson, Victoria M Catterson, Aris L Dimeas, Nikos D Hatziargyriou, Ferdi-nanda Ponci, and Toshihisa Funabashi. Multi-agent systems for power engineering applications part ii: technolo-gies, standards, and tools for building multi-agent systems. Power Systems, IEEE Transactions on, 22(4):1753–1759, 2007.

[10] Mariano Ventosa, Alvaro Baıllo, Andres Ramos, and Michel Rivier. Electricity market modeling trends. Energypolicy, 33(7):897–913, 2005.

[11] Chris Marnay and Owen C Bailey. The certs microgrid and the future of the macrogrid. page 14, 2004.[12] Hurricane Sandy Rebuilding Task Force. Hurricane sandy rebuilding strategy: Stronger communities, a resilent

region, August 2013.[13] Merrill Smith. Us department of energy’s research & development activities on microgrid technologies. In

Vancouver 2010 symposium on microgrids, 2010.[14] Robert H Lasseter. Microgrids and distributed generation. Journal of Energy Engineering, 133(3):144–149, 2007.[15] Nikos Hatziargyriou, Hiroshi Asano, Reza Iravani, and Chris Marnay. Microgrids. Power and Energy Magazine,

IEEE, 5(4):78–94, 2007.[16] Steve Bossart. Doe perspective on microgrids. In Advanced Microgrids Concepts and Technologies Workshop,

2012.[17] Huang Jiayi, Jiang Chuanwen, and Xu Rong. A review on distributed energy resources and microgrid. Renewable

and Sustainable Energy Reviews, 12(9):2472–2483, 2008.[18] Ramon Zamora and Anurag K Srivastava. Controls for microgrids with storage: Review, challenges, and research

needs. Renewable and Sustainable Energy Reviews, 14(7):2009–2018, 2010.[19] NWA Lidula and AD Rajapakse. Microgrids research: A review of experimental microgrids and test systems.

Renewable and Sustainable Energy Reviews, 15(1):186–202, 2011.[20] Wei Huang, Chang-hui Sun, Zi-ping Wu, and Jian-hua Zhang. A review on microgrid technology containing

distributed generation system [j]. Power System Technology, 9:006, 2009.[21] Taha Selim Ustun, Cagil Ozansoy, and Aladin Zayegh. Recent developments in microgrids and example cases

around the world—a review. Renewable and Sustainable Energy Reviews, 15(8):4030–4041, 2011.[22] CM Colson and MH Nehrir. A review of challenges to real-time power management of microgrids. In Power &

Energy Society General Meeting, 2009. PES’09. IEEE, pages 1–8. IEEE, 2009.20

[23] Michael Wooldridge and Nicholas R Jennings. Intelligent agents: Theory and practice. The knowledge engineer-ing review, 10(02):115–152, 1995.

[24] Stuart Jonathan Russell, Peter Norvig, John F Canny, Jitendra M Malik, and Douglas D Edwards. Artificialintelligence: a modern approach, volume 2. Prentice hall Englewood Cliffs, 1995.

[25] Katia P Sycara. Multiagent systems. AI magazine, 19(2):79, 1998.[26] Peter Stone and Manuela Veloso. Multiagent systems: A survey from a machine learning perspective. Autonomous

Robots, 8(3):345–383, 2000.[27] Aris Dimeas and Nikos Hatziargyriou. A multi-agent system for microgrids. In Methods and applications of

artificial intelligence, pages 447–455. Springer, 2004.[28] AL Dimeas and ND Hatziargyriou. Multi-agent reinforcement learning for microgrids. In Power and Energy

Society General Meeting, 2010 IEEE, pages 1–8. IEEE, 2010.[29] Yinliang Xu and Wenxin Liu. Novel multiagent based load restoration algorithm for microgrids. Smart Grid,

IEEE Transactions on, 2(1):152–161, 2011.[30] AL Dimeas and ND Hatziargyriou. Control agents for real microgrids. In Intelligent System Applications to Power

Systems, 2009. ISAP’09. 15th International Conference on, pages 1–5. IEEE, 2009.[31] Zhenhua Jiang. Agent-based control framework for distributed energy resources microgrids. In Intelligent Agent

Technology, 2006. IAT’06. IEEE/WIC/ACM International Conference On, pages 646–652. IEEE, 2006.[32] J Oyarzabal, J Jimeno, J Ruela, A Engler, and C Hardt. Agent based micro grid management system. In Future

Power Systems, 2005 International Conference on, pages 6–pp. IEEE, 2005.[33] Massimo Cossentino, Carmelo Lodato, Salvatore Lopes, Marcello Pucci, Gianpaolo Vitale, and Maurizio Cirrin-

cione. A multi-agent architecture for simulating and managing microgrids. In Computer Science and InformationSystems (FedCSIS), 2011 Federated Conference on, pages 619–622. IEEE, 2011.

[34] Zhu Wang, Rui Yang, and Lingfeng Wang. Intelligent multi-agent control for integrated building and micro-gridsystems. In Innovative Smart Grid Technologies (ISGT), 2011 IEEE PES, pages 1–7. IEEE, 2011.

[35] Hany E Farag, EF El-Saadany, and L El Chaar. A multilayer control framework for distribution systems with highdg penetration. In Innovations in Information Technology (IIT), 2011 International Conference on, pages 94–99.IEEE, 2011.

[36] Zhe Xiao, Tinghua Li, Ming Huang, Jihong Shi, Jingjing Yang, Jiang Yu, and Wei Wu. Hierarchical mas basedcontrol strategy for microgrid. Energies, 3(9):1622–1638, 2010.

[37] AL Dimeas and ND Hatziargyriou. A mas architecture for microgrids control. In Intelligent Systems Applicationto Power Systems, 2005. Proceedings of the 13th International Conference on, pages 5–pp. IEEE, 2005.

[38] SJ Chatzivasiliadis, ND Hatziargyriou, and AL Dimeas. Development of an agent based intelligent control systemfor microgrids. In Power and Energy Society General Meeting-Conversion and Delivery of Electrical Energy inthe 21st Century, 2008 IEEE, pages 1–6. IEEE, 2008.

[39] Aris L Dimeas and Nikos D Hatziargyriou. Operation of a multiagent system for microgrid control. PowerSystems, IEEE Transactions on, 20(3):1447–1455, 2005.

[40] HSVS Kumar Nunna and Suryanarayana Doolla. Multiagent-based distributed-energy-resource management forintelligent microgrids. Industrial Electronics, IEEE Transactions on, 60(4):1678–1687, 2013.

[41] Guping Zheng and Nanfang Li. Multi-agent based control system for multi-microgrids. In Computational Intel-ligence and Software Engineering (CiSE), 2010 International Conference on, pages 1–4. IEEE, 2010.

[42] Evangelos L Karfopoulos, Panagiotis Papadopoulos, Spyros Skarvelis-Kazakos, Inaki Grau, Liana M Cipcigan,Nikos Hatziargyriou, and Nick Jenkins. Introducing electric vehicles in the microgrids concept. In IntelligentSystem Application to Power Systems (ISAP), 2011 16th International Conference on, pages 1–6. IEEE, 2011.

[43] T Logenthiran, Dipti Srinivasan, and Ashwin M Khambadkone. Multi-agent system for energy resource schedul-ing of integrated microgrids in a distributed system. Electric Power Systems Research, 81(1):138–148, 2011.

[44] Chris Colson, MH Nehrir, and RW Gunderson. Multi-agent microgrid power management. In 18th IFAC worldcongress, pages 3678–3683, 2011.

[45] Foo YS Eddy and HB Gooi. Multi-agent system for optimization of microgrids. In Power Electronics and ECCEAsia (ICPE & ECCE), 2011 IEEE 8th International Conference on, pages 2374–2381. IEEE, 2011.

[46] Zhaoyu Wang, Bokan Chen, Jianhui Wang, Miroslav M Begovic, and Chen Chen. Coordinated energy manage-ment of networked microgrids in distribution systems.

[47] Thillainathan Logenthiran, Dipti Srinivasan, Ashwin M Khambadkone, and Htay Nwe Aung. Multiagent systemfor real-time operation of a microgrid in real-time digital simulator. Smart Grid, IEEE Transactions on, 3(2):925–933, 2012.

[48] Zeng Jun, Liu Junfeng, Wu Jie, and HW Ngan. A multi-agent solution to energy management in hybrid renewableenergy generation system. Renewable Energy, 36(5):1352–1363, 2011.

[49] Pierre-Michel Ricordel and Yves Demazeau. From analysis to deployment: A multi-agent platform survey. InEngineering societies in the agents world, pages 93–105. Springer, 2000.

[50] Elhadi Shakshuki and Yang Jun. Multi-agent development toolkits: An evaluation. In Innovations in Applied

21

Artificial Intelligence, pages 209–218. Springer, 2004.[51] Steven F Railsback, Steven L Lytinen, and Stephen K Jackson. Agent-based simulation platforms: Review and

development recommendations. Simulation, 82(9):609–623, 2006.[52] G Nguyen, T Tung Dang, L Hluchy, Z Balogh, M Laclavik, and I Budinska. Agent platform evaluation and

comparison. Rapport technique, Institute of Informatics, Bratislava, Slovakia, 2002.[53] Fabio Bellifemine, Agostino Poggi, and Giovanni Rimassa. Developing multi-agent systems with a fipa-compliant

agent framework. Software-Practice and Experience, 31(2):103–128, 2001.[54] Foundation for Intelligent Physical Agents. Volttron agent examples. http://www.fipa.org/, 1996. [Online;

accessed 29-Jun-2015].[55] Fabio Bellifemine, Agostino Poggi, and Giovanni Rimassa. Jade–a fipa-compliant agent framework. In Proceed-

ings of PAAM, volume 99, page 33. London, 1999.[56] TILAB. Jade. http://jade.tilab.com/, 1996. [Online; accessed 29-Jun-2015].[57] Charles R Robinson, Peter Mendham, and Tim Clarke. Macsimjx: A tool for enabling agent modelling with

simulink using jade. 2010.[58] T Logenthiran, Dipti Srinivasan, and David Wong. Multi-agent coordination for der in microgrid. In Sustainable

Energy Technologies, 2008. ICSET 2008. IEEE International Conference on, pages 77–82. IEEE, 2008.[59] Kehe Wu and Huan Zhou. A multi-agent-based energy-coordination control system for grid-connected large-scale

wind–photovoltaic energy storage power-generation units. Solar Energy, 107:245–259, 2014.[60] HB Gooi, SX Chen, et al. Multi-agent system for distributed management of microgrids. 2015.[61] Murali Krishna Kouluri and RK Pandey. Intelligent agent based micro grid control. In Intelligent Agent and

Multi-Agent Systems (IAMA), 2011 2nd International Conference on, pages 62–66. IEEE, 2011.[62] HN Aung, AM Khambadkone, D Srinivasan, and T Logenthiran. Agent-based intelligent control for real-time

operation of a microgrid. In Power Electronics, Drives and Energy Systems (PEDES) & 2010 Power India, 2010Joint International Conference on, pages 1–6. IEEE, 2010.

[63] Sergio Rivera, Amro M Farid, and Kamal Youcef-Toumi. A multi-agent system transient stability platform forresilient self-healing operation of multiple microgrids. In Innovative Smart Grid Technologies Conference (ISGT),2014 IEEE PES, pages 1–5. IEEE, 2014.

[64] Darith Leng and Sompob Polmai. Control of a micro-grid based on distributed cooperative control of multi-agentsystem.

[65] Luis Gomes, Tiago Pinto, Pedro Faria, and Zita Vale. Distributed intelligent management of microgrids using amulti-agent simulation platform. In Intelligent Agents (IA), 2014 IEEE Symposium on, pages 1–7. IEEE, 2014.

[66] Hyacinth S Nwana, Divine T Ndumu, Lyndon C Lee, and Jaron C Collis. Zeus: a toolkit and approach forbuilding distributed multi-agent systems. In Proceedings of the third annual conference on Autonomous Agents,pages 360–361. ACM, 1999.

[67] British Telecom Intelligent System Research laboratory. Zeus agent development platform. http://

sourceforge.net/projects/zeusagent/, 1999. [Online; accessed 29-Jun-2015].[68] Tinghua Li, Zhe Xiao, Ming Huang, Jiang Yu, and Jingsong Hu. Control system simulation of microgrid based

on ip and multi-agent. In 2010 International Conference on Information, Networking and Automation (ICINA),Kunming, China, pages 235–239, 2010.

[69] Bora Akyol, Jereme Haack, Brandon Carpenter, Selim Ciraci, Maria Vlachopoulou, and Cody Tews. Volttron: Anagent execution platform for the electric power system. In Third International Workshop on Agent Technologiesfor Energy Systems Valencia, Spain, 2012.

[70] PNNL. Volttron agent examples. https://github.com/VOLTTRON/volttron/tree/2.x/Agents, 2014.[Online; accessed 29-Jun-2015].

[71] IDA Modbus. Modbus application protocol specification v1. 1a. North Grafton, Massachusetts (www. modbus.org/specs. php), 2004.

[72] Steven T Bushby. Bacnet tm: a standard communication infrastructure for intelligent buildings. Automation inConstruction, 6(5):529–540, 1997.

[73] IBM Tokyo Research Laboratory. Aglets. http://aglets.sourceforge.net/, 2004. [Online; accessed29-Jun-2015].

[74] David Camacho, Ricardo Aler, Daniel Borrajo, and Jose M. Molina. A multi-agent architecture for intelligentgathering systems. AI Commun., 18(1):15–32, January 2005.

[75] T Sanchez Lopez, Alexandra Brintrup, Duncan McFarlane, and Douglas Dwyer. Selecting a multi-agent systemdevelopment tool for industrial applications: a case study of self-serving aircraft assets. In Digital Ecosystemsand Technologies (DEST), 2010 4th IEEE International Conference on, pages 400–405. IEEE, 2010.

[76] Mariusz Nowostawski, Geoff Bush, Martin Purvis, and Stephen Cranefield. Platforms for agent-oriented software.2000.

[77] Pacific Northwest National Laboratory. Gridlab-d. http://www.gridlabd.org/, 2012. [Online; accessed29-Jun-2015].

22

[78] Robert Lasseter, Abbas Akhil, Chris Marnay, John Stephens, Jeff Dagle, Ross Guttromson, A Sakis Meliopoulous,Robert Yinger, and Joe Eto. Integration of distributed energy resources. the certs microgrid concept. 2002.

[79] Fredrik Ygge and Hans Akkermans. Decentralized markets versus central control: A comparative study. Journalof Artificial Intelligence Research, 11:301–333, 1999.

[80] Michael North, Guenter Conzelmann, Vladimir Koritarov, Charles Macal, Prakash Thimmapuram, and ThomasVeselka. E-laboratories: agent-based modeling of electricity markets. In 2002 American Power Conference, pages15–17, 2002.

[81] Anke Weidlich and Daniel Veit. A critical survey of agent-based wholesale electricity market models. EnergyEconomics, 30(4):1728–1759, 2008.

[82] Rui Duan and Geert Deconinck. Multi-agent coordination in market environment for future electricity infrastruc-ture based on microgrids. In Systems, Man and Cybernetics, 2009. SMC 2009. IEEE International Conferenceon, pages 3959–3964. IEEE, 2009.

[83] Isabel Praca, Carlos Ramos, Zita Vale, and Manuel Cordeiro. Mascem: a multiagent system that simulatescompetitive electricity markets. Intelligent Systems, IEEE, 18(6):54–60, 2003.

[84] T Funabashi, T Tanabe, T Nagata, and R Yokoyama. An autonomous agent for reliable operation of power marketand systems including microgrids. In Electric Utility Deregulation and Restructuring and Power Technologies,2008. DRPT 2008. Third International Conference on, pages 173–177. IEEE, 2008.

[85] Christopher M Colson and M Hashem Nehrir. Comprehensive real-time microgrid power management and controlwith distributed agents. Smart Grid, IEEE Transactions on, 4(1):617–627, 2013.

[86] Christopher M Colson, M Hashem Nehrir, Ratnesh K Sharma, and Babak Asghari. Improving sustainability ofhybrid energy systems part ii: Managing multiple objectives with a multiagent system. Sustainable Energy, IEEETransactions on, 5(1):46–54, 2014.

[87] Perukrishnen Vytelingum, Thomas D Voice, Sarvapali D Ramchurn, Alex Rogers, and Nicholas R Jennings.Agent-based micro-storage management for the smart grid. In Proceedings of the 9th International Conferenceon Autonomous Agents and Multiagent Systems: volume 1-Volume 1, pages 39–46. International Foundation forAutonomous Agents and Multiagent Systems, 2010.

[88] M Nazif Faqiry, Rupam Kundu, Rohan Mukherjee, Sanjoy Das, and Anil Pahwa. Game theoretic model of energytrading strategies at equilibrium in microgrids. In North American Power Symposium (NAPS), 2014, pages 1–4.IEEE, 2014.

[89] E Trinklein, G Parker, W Weaver, R Robinett, L Gauchia Babe, C-W Ten, W Bower, S Glover, and BukowskiS. Scoping study: Networked microgrids,. Technical Report SAND2014-17718, Sandia National Laboratories,2014.

[90] MPF Hommelberg, CJ Warmer, IG Kamphuis, JK Kok, and GJ Schaeffer. Distributed control concepts usingmulti-agent technology and automatic markets: An indispensable feature of smart power grids. In Power Engi-neering Society General Meeting, 2007. IEEE, pages 1–7. IEEE, 2007.

[91] Derek W Bunn and Fernando S Oliveira. Evaluating individual market power in electricity markets via agent-based simulation. Annals of Operations Research, 121(1-4):57–77, 2003.

[92] Frank Sensfuß, Mario Ragwitz, Massimo Genoese, and Dominik Most. Agent-based simulation of electricitymarkets: a literature review. Technical report, Working paper sustainability and innovation, 2007.

[93] Hak-Man Kim, Tetsuo Kinoshita, and Myong-Chul Shin. A multiagent system for autonomous operation ofislanded microgrids based on a power market environment. Energies, 3(12):1972–1990, 2010.

[94] Elaheh Mashhour and SM Moghaddas-Tafreshi. A review on operation of micro grids and virtual power plantsin the power markets. In Adaptive Science & Technology, 2009. ICAST 2009. 2nd International Conference on,pages 273–277. IEEE, 2009.

[95] Hak-Man Kim and T Kinoshita. Multiagent system for microgrid operation based on power market environment.In Telecommunications Energy Conference, 2009. INTELEC 2009. 31st International, pages 1–5. IEEE, 2009.

[96] Hak-Man Kim, Yujin Lim, and Tetsuo Kinoshita. An intelligent multiagent system for autonomous microgridoperation. Energies, 5(9):3347–3362, 2012.

[97] Mohammad Ali Fotouhi Ghazvini, Reza Abedini, Tiago Pinto, and Zita Vale. Multiagent system architecture forshort-term operation of integrated microgrids. 2014.

[98] T Nagata and K Okamoto. Electric power interchange between micro-grids by using multi-agent approach. InAutomation Science and Engineering (CASE), 2014 IEEE International Conference on, pages 50–55. IEEE, 2014.

[99] Takeshi Nagata and Kazuya Okamoto. A multi-agent based optimal operation for microgrid. In Systems, Manand Cybernetics (SMC), 2014 IEEE International Conference on, pages 3791–3796. IEEE, 2014.

[100] Bo Zhao, Meidong Xue, Xuesong Zhang, Caisheng Wang, and Junhui Zhao. An mas based energy managementsystem for a stand-alone microgrid at high altitude. Applied Energy, 143:251–261, 2015.

[101] J Duncan Glover, Mulukutla Sarma, and Thomas Overbye. Power System Analysis&Design, SI Version. CengageLearning, 2011.

[102] Jiang Wei, He Zheng-you, and Bo Zhi-qian. The overview of research on microgrid protection development. In

23

Intelligent System Design and Engineering Application (ISDEA), 2010 International Conference on, volume 2,pages 692–697. IEEE, 2010.

[103] Benjamin Kroposki, Robert Lasseter, Toshifumi Ise, Satoshi Morozumi, S Papatlianassiou, and Nikos Hatziar-gyriou. Making microgrids work. Power and Energy Magazine, IEEE, 6(3):40–53, 2008.

[104] Seetaram Alwala, Ali Feliachi, and MA Choudhry. Multi agent system based fault location and isolation in asmart microgrid system. In Innovative Smart Grid Technologies (ISGT), 2012 IEEE PES, pages 1–4. IEEE, 2012.

[105] M Pipattanasomporn, H Feroze, and S Rahman. Multi-agent systems in a distributed smart grid: Design andimplementation. In Power Systems Conference and Exposition, 2009. PSCE’09. IEEE/PES, pages 1–8. IEEE,2009.

[106] C Ma, Z Lu, WH Tang, QH Wu, and J Fitch. An agent brokering-based scheme for anti-islanding protectionof distributed generation. In Power & Energy Society General Meeting, 2009. PES’09. IEEE, pages 1–8. IEEE,2009.

[107] Zina Boussaada, Octavian Curea, Haritza Camblong, Najiba Bellaaj Mrabet, and Amelie Hacala. Multi-agentsystems for the dependability and safety of microgrids. International Journal on Interactive Design and Manu-facturing (IJIDeM), pages 1–13, 2014.

[108] Zhi Wu and Wei Gu. Active power and frequency control of islanded microgrid based on multi-agent technology[j]. Electric Power Automation Equipment, 11:57–61, 2009.

[109] Zhang Jian, Ai Qian, Jiang Chuanwen, Wang Xingang, Zheng Zhanghua, and Gu Chenghong. The applicationof multi agent system in microgrid coordination control. In Sustainable Power Generation and Supply, 2009.SUPERGEN’09. International Conference on, pages 1–6. IEEE, 2009.

[110] Meiqin Mao, Peng Jin, Nikos D Hatziargyriou, and Liuchen Chang. Multiagent-based hybrid energy managementsystem for microgrids. 2014.

[111] Takumi Kato, Hideyuki Takahashi, Kazuto Sasai, Gen Kitagata, Hak-Man Kim, and Tetsuo Kinoshita. Priority-based hierarchical operational management for multiagent-based microgrids. Energies, 7(4):2051–2078, 2014.

[112] AL Kulasekera, RARC Gopura, KTMU Hemapala, N Perera, and Achala Pallegedara. Dual layered multi agentsystem for intentional islanding operation of microgrids. 2012.

[113] Xiaojing Chen, Bing Kong, Fan Liu, Xue Gong, and Xia Shen. System service restoration of distribution networkbased on multi-agent technology. In Digital Manufacturing and Automation (ICDMA), 2013 Fourth InternationalConference on, pages 1371–1374. IEEE, 2013.

[114] I-H Lim, Tarlochan S Sidhu, M-S Choi, S-J Lee, Sugwon Hong, S-I Lim, and S-W Lee. Design and implementa-tion of multiagent-based distributed restoration system in das. 2013.

[115] Jignesh M Solanki, Sarika Khushalani, and Noel N Schulz. A multi-agent solution to distribution systems restora-tion. Power Systems, IEEE Transactions on, 22(3):1026–1034, 2007.

[116] YL Lo, CH Wang, and CN Lu. A multi-agent based service restoration in distribution network with distributedgenerations. In Intelligent System Applications to Power Systems, 2009. ISAP’09. 15th International Conferenceon, pages 1–5. IEEE, 2009.

[117] CM Colson, MH Nehrir, and RW Gunderson. Distributed multi-agent microgrids: a decentralized approach toresilient power system self-healing. In Resilient Control Systems (ISRCS), 2011 4th International Symposium on,pages 83–88. IEEE, 2011.

[118] Peng Li, Bin Song, Wei Wang, and Tiemin Wang. Multi-agent approach for service restoration of microgrid. InIndustrial Electronics and Applications (ICIEA), 2010 the 5th IEEE Conference on, pages 962–966. IEEE, 2010.

[119] Hao Liang, Bong Jun Choi, Weihua Zhuang, Xuemin Shen, ASA Awad, and A Abdr. Multiagent coordination inmicrogrids via wireless networks. Wireless Communications, IEEE, 19(3):14–22, 2012.

[120] Niannian Cai, Xufeng Xu, and Joydeep Mitra. A hierarchical multi-agent control scheme for a black start-capablemicrogrid. In Power and Energy Society General Meeting, 2011 IEEE, pages 1–7. IEEE, 2011.

[121] Sridhar Chouhan, Jawad Ghorbani, Hakan Inan, Ali Feliachi, and Muhammad A Choudhry. Smart mas restorationfor distribution system with microgrids. In Power and Energy Society General Meeting (PES), 2013 IEEE. IEEE,2013.

[122] Michael Wooldridge and Nicholas R Jennings. Pitfalls of agent-oriented development. In Proceedings of thesecond international conference on Autonomous agents, pages 385–391. ACM, 1998.

[123] Nicholas R Jennings. On agent-based software engineering. Artificial intelligence, 117(2):277–296, 2000.[124] Omer F Rana and Kate Stout. What is scalability in multi-agent systems? In Proceedings of the fourth interna-

tional conference on Autonomous agents, pages 56–63. ACM, 2000.[125] David Manz, Reigh Walling, Nate Miller, Beth LaRose, Rob D’Aquila, and Bahman Daryanian. The grid of the

future: Ten trends that will shape the grid over the next decade. Power and Energy Magazine, IEEE, 12(3):26–36,2014.

[126] Luis Hernandez, Carlos Baladron, Javier M Aguiar, Belen Carro, Antonio J Sanchez-Esguevillas, Jaime Lloret,and Joaquim Massana. a survey on electric power demand forecasting: Future trends in smart grids, microgridsand smart buildings. Communications Surveys & Tutorials, IEEE, 16(3):1460–1495, 2014.

24

[127] Amro M Farid. Multi-agent system design principles for resilient coordination & control of future power systems.Intelligent Industrial Systems, pages 1–15, 2015.

25