Intelligent Operation and Control of Microgrids Using Multiple Reinforcement

Learning Agents

Raja Suryadevara

Penn State Harrisburg

Master of Science in Electrical Engineering

Advisor: Peter B. Idowu

Funding Source: Department of Defense, Office of Naval Research (DoD-ONR)

Introduction

Smart Grids

Figure 1: Smart Grid Technology [1]

Research Purpose

• Distributed Control

• Solve problems with unpredictability using AI

• Reduce Costs and Pollution

• Maximize Grid Efficiency

• Maintain Power Quality

Centralized vs Distributed Generation

Central Power

Plant

Commercial Consumers

Industrial Consumers

Residential Consumers

Wind Farm

Fuel Cell

Industrial

Consumers

Residential

Consumers

Commercial

Consumers

Commercial

Consumers

Solar

Power

Commercial

Consumers

Residential

Consumers

Solar

Power

Solar

Power

Central

Power Plant

Figure 2: Centralized Generation [1] Figure 3: Distributed Generation [1]

Hardware Implementation

Figure 4: Experimental Hardware Setup

Communications Network

Protocols:

1) TCP/IP

2) MODBUS

3) OPC UA

4) IEC 61850

Figure 5: Communications Network

Human Machine Interface (HMI)

Figure 6: Human Machine Interface

Methodology

• Deep Reinforcement Learning

Formulate

Problem

Create

EnvironmentDefine

Reward

Create

Agent

Train

Agent

Validate

Agent

Deploy

Policy

Check

Results

Figure 8: Training Workflow

Figure 7: Single RL Agent Figure 9: Multi Agent Network

Frequency ControlReward Signal

Stop Signal

RL Agent

Figure 10: Frequency Control Agent

Results

Conclusion

• Reliable Communications Infrastructure

• Satisfactory results with frequency control

• Motivation towards multi-agent network

• Long training time

Future Improvements

PARALLEL

COMPUTING

GPU ACCELERATION OPTIMIZE LEARNING

ALGORITHM

References

[1] P. Idowu and R. Suryadevara “Hardware-based microgrid testbed to facilitate development of Distributed Energy Resource (DER) systems for sustainable growth”, ICMA-SURE 2020.

[2] “PPL Electric Utilities Power Lab”, Penn State Harrisburg, Department of Electrical Engineering. [Online]. Available: https://sites.psu.edu/microgridtestbedpsh/.

[3] U.S. Department of Energy, Office of Electricity Delivery and Energy Reliability, “The Smart Grid: An Introduction,” 2008.

[4] Sutton, R.S., and A.G. Barto. Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning series. MIT Press, 2018. Available: https://books.google.de/books?id=6DKPtQEACAAJ.