A Low-cost and Scalable Visualization System for Electricity Consumption

A Low-cost and Scalable Visualization System

for Electricity Consumption

Ryousei Takano

Information Technology Research Institute, National Institute of Advanced Industrial Science and Technology (AIST),

Japan

SC2011@Seattle, Nov. 17 2011

2

Outline

•  Background •  System overview •  Demonstration at SC11 •  Summary

Background

3

•  The power consumption of data centers and networks becomes an issue of vital importance to IT industries.

•  In Japan, we faced planed

power outages (rolling power outages) due to power shortages caused by 3.11.

Fukushima Daiichi Nuclear Power Plant

Google data center in the Dalles, Oregon

Motivation •  To plan electricity savings, visualization of electricity

consumption per segmented unit, such as a power distribution board, is becoming essential.

•  But, the total system cost and scalability are problem. –  a lot of distributed sensors...

•  The system has to be low-cost, scalable, and ease to develop applications. ➡ Cheap power measuring hardware unit ➡ Power logging software utilizing cloud computing ➡ A simple REST protocol

4

System overview

5

…

…

Data collecting unit

Viewer

Alert

Google App Engine

Datastore

Update power usage using REST w/ JSON Retrieve data using

REST w/ JSON


1

2

3

4

Data measuring unit


Applications

Data collecting server

Observe the state of power consumption Plan electricity saving

Small start Go big

6

4 sensors 32 ports = 128 sensors

Google App Engine

Datastore

...

Google App Engine

Datastore

Sensors can be incrementally installed, GAE can scale up according to the workload.

Low-cost power measuring unit

7

RJ-45 port

Signal processing board (dsPIC30F3013)

Clamp-on current sensor (max: 4)

•  Send data to data collecting unit every second. •  The production cost is approximately 120 USD,

including the cost of 4 current sensors.

Data collecting unit (1/2)

8

•  Gather data from up to 32 power measuring unit •  Push data to GAE

–  Can be placed behind NAT

To power measuring unit (Not Ethernet, data transfer and power supply)

To GAE via the Internet (Ethernet/100BaseT)

Data collecting unit (2/2)

9

RJ-45 ports x 32

Power measuring unit

CPU (SH3) board - Linux + Python

Serial-Parallel signal transform (Now, it has been replaced by FPGA.)

GAE

Google App Engine

10

•  PaaS cloud service for web applications –  Java, Python, Go are supported –  Your application will have URL like

http://XXXX.appspot.com •  Scalable and stable data storage

–  Data are replicated to 5 different datacenters –  Allows 2 of them to be lost during operation

•  Maintenance free –  No need to manage, almost

•  Almost free, for now –  Price change is scheduled this Dec. though.

REST API

11

path method description

/update POST Upload data

/latest GET Get all data for the last minute

/latest,N GET Get all data for the last N minutes

/summary.s/YYYYmmDDHHMMSS,N GET Get all data for each second start from YYYYmmDDHHMMSS, for N seconds

/summary.m/YYYYmmDDHHMM,N GET Get all data for each minute start from YYYYmmDDHHMM, for N minutes

/summary.h/YYYYmmDDHH,N GET Get all data for each hour start from YYYYmmDDHH, for N hours

/summary.d/YYYYmmDD,N GET Get all data for each day start from YYYYmmDD, for N days

/query.s/LOC/YYYYmmDDHHMMSS,N GET Get data for locations that name start with LOC

/query.m/LOC/YYYYmmDDHHMM,N GET

/unit-config/UNIT_ID GET Get configuration data

/unit-config/UNIT_ID PUT Set configuration data

Update from Data collecting unit Each data collection unit sends data every 20 seconds

–  POST the following JSON string

12

{ "id": "UNIT_ID" "time": "1319837460” /* elapsed seconds from the UNIX epoch time */ "power": { /* data for the last 20 seconds per measurement point */ "sensor0.0": [VAL0, VAL1, VAL2, VA3, ..., VAL19], "sensor0.1": [VAL0, VAL1, VAL2, VA3, ..., VAL19], "sensor1.0": [VAL0, VAL1, VAL2, VA3, ..., VAL19], .... } }

GAE xxx.appspot.com/update

Data retrieval

13

GAE xxx.appspot.com/latest,N

{ "time": “1319837460” /* epoch time */ "timeStr": “201110290631” /* human readable time in JST */ "power": { "LOCATION0": [1234] "LOCATION1": [1234] "LOCATION2": [1234] "LOCATION3": [1234] "LOCATION4": [1234] ... } }

Viewer application

Applications periodically (e.g., 1 min) get data from GAE

–  GET the following JSON string

Demonstration at SC11

14

…

…

AIST (JP) Data collecting

unit

7 data collecting units 82 sensor modules 290 measurement points

Sensor module Clamp-on current transformer

Data collection unit

NICT booth #4717(US)

2 sensor built-in power strips

Viewer

Viewer

Google App Engine

Datastore

Update power usage using REST w/ JSON Retrieve data using

REST w/ JSON


AIST booth #917(US)

1

1

2

2 3

1

2

3

gather

ｓtore

view

15 Power measuring unit

Clamp-on current sensor

Power distribution board

GAE

AIST server room (Japan)

Data collecting unit in free access floor

NICT booth #4717 (US)

16

2 sensor modules (sensor built-in power strip)

Data collecting module

Tiled display wall server

Juniper’s routers

GAE

Visualization App.

17

AIST server room (ACS cluster) at Japan

NICT booth at the US

Summary •  Our newly developed system helps reduce total system

cost and improve scalability by employing low-cost power measuring units (30 USD per measurement point), and utilizing cloud computing. –  The development of the system was completed within 3 months. –  At SC11, we have successfully demonstrated visualization of

power data gathering from sensors installed at the US and Japan.

•  Future work: Incorporate into reactive/dynamic server consolidation on a data center to reduce power consumption aggressively.

18

Q&A

19

Thank NICT members for their help in the demonstration.""This research was partially supported by the NEDO research project entitled “Research and Development Project for Green Network/System Technology (Green IT Project).”"

Thanks for your attention!

Demo: GAE dashboard

20

0.75 HTTP requests/second

5000 bytes received/second

Visualization Applications

21

less than 90% of the upper limit less than 95% of the upper limit more than 95% of the upper limit Offline

(a) Web application (b) Desktop application