Power Systems Hardware-in-the-Loop Laboratory Testbed …

MIT Lincoln Laboratory

This work is sponsored by the Department of Energy, Office of Electricity Delivery and Energy Reliability, under Air Force Contract #FA8721-05-C-0002. Opinions,

interpretations, conclusions and recommendations are those of the author and are not necessarily endorsed by the United States Government.

DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.

Erik Limpaecher

Power Systems Hardware-in-the-Loop

Laboratory Testbed and Open Platform

(HILLTOP)

February 16, 2017

DOE HILLTOP - 2

ERL 22 February 2017

DOE HILLTOP - 3


DOE HILLTOP - 4


Integration Test Example C

urr

en

t V

olt

ag

e

Data capture: Scott Manson, SEL

Breaker opens

Fault

Load shedding Generator

control response Short circuit

DOE HILLTOP - 5


DOE HILLTOP - 6


Microgrid Test Feeders

“This isn’t rocket science; it’s way more complicated.”

– Ed Corbett

“…and more important.” – Erik Limpaecher

Emergent behavior is behavior of a system that does not

depend on its individual parts, but on their relationships to

one another. Thus emergent behavior cannot be predicted

by examination of a system's individual parts. It can only

be predicted, managed, or controlled by understanding the

parts and their relationships.

DOE HILLTOP - 7


• High NRE for each project

– One vendor’s microgrid controller quote: $1M starting price

• “Vaporware”

– No standard list of functions or performance criteria

– Difficult to validate marketing claims

• Risk of damage to expensive equipment

– One utility-deployed microgrid: 1 year of controls testing, damaged a 750 kW transformer, required significant engineering staff support

• Interconnection behavior unknowable to utility engineers

– Controls are implemented in proprietary software

– Microgrids are a system of systems: Exhibit emergent behavior

• No standards verification

– IEEE P2030.7 and P2030.8 standards are on the horizon

How To Accelerate Advanced Distribution System Deployment?

Need to reduce integration time, cost, & risk

DOE HILLTOP - 8


Types of Power System Testbeds

Full System

Inv G

C C

Inv

Power Testbed

G

C C

Power HIL

Inv

C

G

C

Controller HIL

C

GInv

C

Simulation

GInv

CC

mC mC mC

DMS DMS

LegendG generatorInv battery or solar inverterC device controllermC microgrid controllerDMS distribution management system controller

power gridhigh-bandwidth AC-AC convertersimulation or emulation boundary

hardwarevirtual (simulated or emulated)

Matlab

SimPowerSystems

simulation

(not real-time)

Florida State CAPS

facility

Princeton University

cogeneration plant and

microgrid

ORNL DECC

Microgrid Lab

MIT-LL HILLTOP

System

DOE HILLTOP - 9


Power Distribution Integration Platforms and Testbeds

Full System

Inv G

C C

Inv

Power Testbed

G

C C

Power HIL

Inv

C

G

C

Controller HIL

C

GInv

C

Simulation

GInv

CC

mC mC mC

DMS DMS

LegendG generatorInv battery or solar inverterC device controllermC microgrid controllerDMS distribution management system controller

power gridhigh-bandwidth AC-AC convertersimulation or emulation boundary

hardwarevirtual (simulated or emulated)

low cost

moderate cost

high cost

Simulation Controller

HIL

Power HIL

Power Testbed

Full System

Te

st

Co

vera

ge

Test Fidelity

Low

Low High

High

DOE HILLTOP - 10


• Microgrid controller HIL simulates in real-time at sub-cycle timescales

Full Test Coverage

Useful for:

• Steady-state

• Dynamic analysis

• Transient analyses

10-5 10-4 10-3

1 ms10-2 10-1 100

1 s

Time (seconds)

10-6

Power converter switching

frequencies(3.5, 5 kHz)

OPAL-RT simulation step

100 ms(10 kHz)

One AC cycle

16.7 ms(60 Hz)

User display update rate

66.7 ms(15 Hz)

Load profile & irradiance data

1 s(1 Hz)

Power converter controller response

0.5-1 ms(1-2 kHz)

Secondary control0.1-1 s

(1-10 Hz)

Genset protection functions

0.01-0.02 s(50-100 Hz)

Power system fault transients

0.3-1 ms(1-3 kHz)

Typhoon HIL simulation step

500 ns(2 MHz)

Power converter data sampling rate

(7-10 kHz)

DOE HILLTOP - 11


• Research & Development

• Standards Compliance Testing

• Partner / Vendor Selection

• Proposal, Business Development

• Feasibility Study, Conceptual Design

• Preliminary & Final Design, Development

• Factory Acceptance Testing

• Commissioning / Field Testing

• Root Cause Investigation

RD&D Cycle for Modern Distribution System Projects

– 2 –

Development

Platform

– 3 –

Deployment

Platform

– 4 –

Standards Test

Platform

– 1 –

Engagement

Platform

DOE HILLTOP - 12


Vision for Power Systems HILLTOP

– 2 –

Development

Platform

– 3 –

Deployment

Platform

– 4 –

Standards Test

Platform

• Cost-effective systems integration and testing

• Decrease risk on “brownfield” sites operating legacy equipment

• Enable performance evaluation of commercial products

• Pre-commission testing of advanced power system projects

• Test edge conditions and exercise the actual device controllers

• Technical risk reduction for electric power utilities

• Industry-standard test platform for new power systems

• Certify to IEEE P2030.8, P1547, and utility interconnection rules

– 5 –

Electric Power HIL Controls Collaborative (EPHCC) Shared Repository

– 1 –

Engagement

Platform

• Provide a tangible proof-of-concept to new project stakeholders

• Accelerate the sales cycle by showing an operating system

• Use for rapid iteration of feasibility studies

• Demonstrations at Microgrid and DER Controller Symposium

DOE HILLTOP - 13


Improvements Since Previous Symposium

Category Improvement

Since 1st Symposium

Real-time test platforms & microgrid controllers 2x

Ported simulation environments 5x

Physical device controllers 4x

Test feeders 3x

Test cases 4x

Total 1000x

In other words +60 dB

DOE HILLTOP - 14


• Motivation

• Testbed Buildup

• Integration Process

• Demonstration Orientation

Outline

DOE HILLTOP - 15


4.5 MW 4 MW 6 MW

Test Feeders

• Representative of a community microgrid

• Supports thorough controller evaluation

– Multiple reconfigurations possible

– Multiple interconnections to the utility

– Insufficient generation when islanded

Peak demand 14 MW

Available generation 10 MW

Software components 100+

DERs 5

Relays ~50

DOE HILLTOP - 16


4.5 MW 4 MW 6 MW

Test System Segmentation for Real-time Simulation

• 3 cores: One per feeder + machine models

• 3 cores: Relay models for each feeder + UDP communications

• 1 core: Storage & PV power electronics models

• 1 core: Utility substation + high-speed data collection

DOE HILLTOP - 17


HILLTOP Rack #1 Using OPAL-RT Simulator

• FPGA-based I/O management with

− Xilinx Spartan-3

• Real-time target with Xeon Intel® Processor

− Using 8 of 12 cores

− 2.7 to 3.2 GHz

DOE HILLTOP - 18


• 4 HIL603 units

– High speed serial link interconnection 8 lane - 5GHz

• 2µs & 4µs time steps

– Multirate electrical simulation

• 23 cores used

• 20ns digital sampling

• 42 simulated relays with Modbus comms

– 1.2 ms execution rate

HILLTOP Testbed #2 Using Typhoon HIL Simulator

HIL603 real-time simulator

Woodward HILConnect

SEL HILConnect

EPC HILConnect (PV and ESS)

DOE HILLTOP - 19


Add Machine and Power Electronics Models

Hardware-in-the-Loop Simulator

Simulated Microgrid Feeder


3.5 MW

4 MVA

4 MVA

R1

R8R7

R5

Gen

R6

PV

Bat

Simulated Microgrid FeederSimulated Microgrid Feeder and Devices

R4

R2R3

Create detailed models of the DER devices

Solar Inverter

Genset Machine Model

Power Electronics Model

Bidirectional Power Converter Power Electronics Model

DOE HILLTOP - 20


Test Stimuli


Load B01 Load B02

Load B03

3.5 MW

4 MVA

4 MVA

R1

R8R7

R5

Gen

M 250 hp460 V

R6

PV

Bat

Interruptible Critical

Priority


Loads

Motors

Irradiance

Grid Status

R4

R2R3

Assign Load Priorities, Add Test Stimuli

DOE HILLTOP - 21


Add Commercial Controllers as Hardware-in-the-Loop

Test Stimuli


Load B01 Load B02

Load B03

3.5 MW

4 MVA

4 MVA

R1

R8R7

R5

Gen

M 250 hp460 V

R6

PV

Bat


Priority


Loads

Motors

Irradiance

Grid Status

R4

R2R3

Physical Device

Controllers

SEL 2440

Fast Load Shed

SEL 751

Relays

EPC PV Inverter

Controller

Woodward

Genset / CHP

Controllers

EPC Power

Converter

Controller

CRelay

DOE HILLTOP - 22


Integrate Microgrid Controllers

Test Stimuli


Load B01 Load B02

Load B03

3.5 MW

4 MVA

4 MVA

R1

R8R7

R5

Gen

M 250 hp460 V

R6

PV

Bat


Priority


Loads

Motors

Irradiance

Grid Status

R4

R2R3

Physical Device

Controllers

CRelay

SEL 2440

Fast Load Shed

SEL 751

Relays

Woodward

Genset / CHP

Controllers

EPC Power

Converter

Controller

EPC PV Inverter

Controller

Microgrid Controller

DOE HILLTOP - 23


Integrate Microgrid Controllers

Test Stimuli


Load B01 Load B02

Load B03

3.5 MW

4 MVA

4 MVA

R1

R8R7

R5

Gen

M 250 hp460 V

R6

PV

Bat


Priority


Loads

Motors

Irradiance

Grid Status

R4

R2R3

Physical Device

Controllers

CRelay

SEL 2440

Fast Load Shed

SEL 751

Relays

Woodward

Genset / CHP

Controllers

EPC Power

Converter

Controller

EPC PV Inverter

Controller


Distribution System Operator /

Dispatcher

DOE HILLTOP - 24


Simulated Diesel Genset Block

1 MW Genset 4 MW Genset

Manufacturer / Model CAT C32 CAT C175-20

Rating (kVA) 1,000 4,000

Power Factor TBD TBD

Voltage (V) 480 13,800

Frequency (Hz) 60 60

Speed (RPM) 1800 1800

Minimum Output Power 25kW 100kW

Startup Time <10 sec <15 sec

Genset ratings and characteristics

Synchronous Machine, Governor, and AVR Models

DOE HILLTOP - 25


• Generator statistics

• High speed instrumentation

– 8kHz sampling

– Voltage and current

– Bias signals

HILLTOP Integration: Model Validation

Shutdown of Generator and Model Start-up of Generator and Model

• 1 MVA

• 480Vac

• 3 phase

DOE HILLTOP - 26


Natural Gas Combined Heat and Power

CHP Aggregate Thermal Model

GE/Jenbacher J620 NG Engine (1800 RPM)

3.5 MW Natural Gas Engine Model (Physics Based)

• Physics-based, scalable 3.5 MW NG genset with gas valve, intake manifold, combustion

– Fuel usage

– GHG emissions

– Heat recovery

– Woodward easYgen 3500 compatible

• Aggregate CHP system model

– Modbus commanded heating or cooling mode, temperature set-point

– Independent heat load input

– Parametrically settable losses, cooling coefficient of performance, thermal inertia

Ele

ctr

ical

Th

erm

al

Valve body Intake

manifold

Engine

CHP Steam

Temperature

control

Set point Heat load

DOE HILLTOP - 27


EPC Power Electronics Models: Solar PV and Energy Storage System

• 4-quadrant control module

• Control capabilities for microgrid operation

– Real (P) and Reactive (Q) power dispatch

– Voltage islanding mode

– UPS parallel backup mode

• Manufacturer validated inverter model

• Modbus over RS485 Communication

DOE HILLTOP - 28


Generic Software Relay

• Used for telemetry

• Various time current characteristic curves (TCC)

• Active protection features:

– Overcurrent (50, 51)

– Over/under voltage (27, 59)

– Synchronism check (25)

– Reclosing (79)

• Modbus TCP interface

• Multiple protection group settings accessible by the microgrid controller

– Grid-tied protection

– Islanded protection

Protection

Functions

Data Logging Registers

Mapping

Preliminary

Calculations

Degree of inverseness

approximating SEL-751

DOE HILLTOP - 29


Device Controller Integration: Woodward easYgen 3500

Interface Box

Woodward easYgen

3000 Controller

Genset Simulation in HIL

Governor

AVR

MMCBGCB

Load

16 Vac 16 Vac

Vab

c

Iab

c

16 Vac

120 Vac

0-1 A

120 VAC

Feed

er

Tach

om

eter

Vab

c

16 Vac

0-1 A

Ia

F_ref: 60 Hz

V_ref: 480 VLL

±5V square wave

Throttle

Field

G

Woodward

easYgen 3500

Genset

Controller

Legend M Motor G Generator GCB Generator Circuit Breaker MCB Mains Circuit Breaker

Signal voltage transformer

Voltage-controlled current source

DOE HILLTOP - 30


Models Ported to Numerous Platforms

SEL RTDS Testbed

Factory Acceptance Test Eaton Protection

Coordination Study

MIT Smart Grid in a Room

Simulator for ARPA-E

Typhoon HIL

HILLTOP testbed

NREL Microgrid Challenge

EPHCC Shared Repository

MIT-LL Test Feeders and

Models

DOE HILLTOP - 31


Today: HILLTOP on 2 Real-time Sims Operating 4 Microgrid Controllers

SEL Microgrid Controller

Schneider Microgrid Controller

Eaton Microgrid Controller

GE Microgrid Controller

DOE HILLTOP - 32


• Motivation

• Testbed Buildup



Outline

DOE HILLTOP - 33


• I/O point check

• Communications

– Comm w/out errors, change setpoints, scale factors

• Device-level performance characterization

– Load acceptance, load rejection tests

– Large setpoint changes

– Determine capability curves

• Customize site-specific controls / DER mode changes

• DER paralleling / generator load sharing

The Integration Process: Device Integration

DOE HILLTOP - 34


The Integration Process: Microgrid Controller Configuration

• Transitions

– Intentional islanding

– Unplanned islanding

– Synchronization and reconnection

• Steady-state operation

– Grid-tied optimal dispatch

– Islanded stability

• Faults

– Protection

– Reconfiguration for critical load service

DOE HILLTOP - 35


• Motivation

• Testbed Buildup



Outline

DOE HILLTOP - 36


Demo #1: SEL Controller on OPAL-RT Focus: Black Start, Islanding, & Fast Load Shedding





10:00am

DOE HILLTOP - 37


#3: Schneider on OPAL-RT Focus: Reconfiguration & Load Prioritization





2:10pm

break

DOE HILLTOP - 38


#4: Eaton Controller on Typhoon HIL Focus: Protection & DER Dispatch





3:10pm

DOE HILLTOP - 39


Demonstration Sequence

DOE HILLTOP - 40


Buses, Breakers, and Loads

LEGEND

Live Bus

Dead Bus

Closed Breaker

Open Breaker

Critical Load

Priority Load

Interruptible Load

Energized Load

Dropped Load

Energized

Motor

Load

DOE HILLTOP - 41


Feeder Status and Power Factor

Grid-tied Feeder

Islanded Feeder

DOE HILLTOP - 42


Distributed Energy Resources

Discharging Energy Storage

System (ESS) / Battery

Energized Solar

PV Array

Energized Diesel Genset

Energized Natural Gas-fired Combined

Heat and Power (CHP) Plant

DOE HILLTOP - 43


Real and Reactive Power Production and Capability Curves

DOE HILLTOP - 44


Testing of Future Advanced DER: IEEE P1547 Draft Amendment

P1547

LEGEND

Nominal

Normal range

Must ride through

May ride through

Must trip

DOE HILLTOP - 45


Energy Sources “Sand Chart”

DOE HILLTOP - 46


High-fidelity Oscilloscope Waveforms

LEGEND

Grid voltage

CHP current

CHP voltage

Diesel genset voltage

DOE HILLTOP - 47


Our Mission: Transitioning Technology for National Energy Resilience

– 2 –

Development

Platform

– 3 –

Deployment

Platform

– 4 –

Standards Test

Platform

• Vendor / new equipment evaluation

• Advanced controls R&D

• Controller integration

• Pre-commissioning testing

• IEEE P2030.8 and P1547 conformance testing

– 5 –

Electric Power HIL Controls Collaborative (EPHCC) Shared Repository

– 1 –

Engagement

Platform

• Proof-of-concepts

• Stakeholder engagement (utilities, regulators, host sites)

• Industry support: model repository

DOE HILLTOP - 48


Sponsor

Dan Ton, DOE OE

MIT Lincoln Laboratory

Reynaldo Salcedo

Ed Corbett

Kendall Nowocin

Chris Smith

Igor Pedan

Liz Dalli

Raajiv Rekha

Joe Cooley

Tammy Santora

Marija Ilic

Scott van Broekhoven

Bill Ross

Acknowledgements

Collaborators

SEL

Schneider Electric

Eaton

General Electric

Typhoon HIL

EPC Power

OPAL-RT

Scott Manson, SEL

Will Allen, SEL

Tom Steber, Schneider

Vijay Bhavaraju, Eaton

Ivan Celanovic, Typhoon HIL

Allan Abela, EPC Power

Galen Nelson, MassCEC

Jessica Ridlen, MassCEC

Fran Cummings, Peregrine Group

Babak Enayati, National Grid

Jim Reilly, Reilly Associates

DOE HILLTOP - 49


Erik Limpaecher

Assistant Group Leader

Energy Systems, Group 73

781-981-4006 (lab)

[email protected]

DOE HILLTOP - 50


BACKUP

DOE HILLTOP - 51


HIL Platform Block Diagram

IEC61850Modbus TCP

Communication

Translation Hardware /

FirmwareCOM

Microgrid Controller (Unit Under Test)

MIT-LL

Interface Box

Vendor-supplied equipment

Modbus RS485

Woodward

EasyGen 3500

#1

COM DIO AIO COM DIO AIO

Simulated

Diesel Genset

Simulated

Combined Heat and

Power System

Simulated

Battery Storage &

Power Converter

Simulated Breakers

Simulated Distribution Grid

and Microgrid Test Feeder

Simulated

PV & Inverter

COM DIO AIOCOM DIO AIOCOM DIO AIO

Simulated

Generic

Relays

Woodward

EasyGen 3500

#2

Modbus RS485

Firewall and Network Switch

Lantronix

Intellibox 2100

TCP to RS485

Lantronix

Intellibox 2100

TCP to RS485

SEL-751 Relays

(qty 3)

Simulated

Load

Meters

EPC Power

Electronics

Controller #1:

Storage

EPC Power

Electronics

Controller #2:

PV

Modbus TCP

MIT-LL

Interface Box

OPAL-RT and

Typhoon HIL

Real-time Simulation

Platforms


Mechanical Enclosure

Cyber Test

C2 Proxy

Load Switches

COM DIO AIO

IEC61850

Fast Load

Shedding

SEL-2440

(Optional)

Update diagram to make text larger, add

DSO

Documents

Power Systems Hardware-in-the-Loop Laboratory Testbed …