Final Report

Smart Grid Demo Table

Modification of communication

Sanni Svärd (625092)

Yoann Chappaz (629455)

Group: S4.1.30

1

School year: 18/19 Semester 4

Introduction 2

Analysis 2

Determining the operating system 3

System of communication 3

Technical design 3

Hardware part 4

New schematic for one table 4

Designing PCB 5

Software part 5

Arduino code 5

MQTT 6

Serial communication 7

Python code 7

Design phase 8

Implementation phase 9

Test report 11

Test setup 11

Analysis 12

Final conclusions 13

2

Introduction

Smart Grid is a demo table to show a “intelligent” electric network. This product is used by

scientists and students to simulate the electrical flow in the real smart city. This enables to

optimize the production, the supply and the storage of energy. In addition, the Smart Grid is

used to reduce greenhouse gas in the smart city and struggle against the global warming.

Smart grid demo table have several table modules. They can be set up with three voltage

range: high, medium and low. At the high voltage table modules are power plants, medium for

example factories and low residents, electric cars, etc. You can add many tables together and

increase the size of the city. Energy can be share toward another voltage range thanks to

transformers. For every range, there are different 3D modules that you can add. Whereas

houses must be connected in low voltage, batteries have not specific voltage. Configuration

of the position of tables is defined before starting the simulation. Adding module to the table

change the flow. That is how you can see how adding or removing energy consumers or

producers will affect to hole entity. Current table parts can be found at HAN university of

applied sciences.

The system is controlled by a central computer called “main controller” which calculates new

flows. This latter communicate with each table thanks to the boss of table controllers. In the

one table there is two microcontrollers : the helper one which sends the information from four

RFID readers to the boss table controller which checks the others 4 RFID-readers,

communicate with main controller and change the state of ledstrips to show the flow of energy.

When adding 3D models to the readers, microcontroller send the information to the main

controller which calculates the new flow and send back new values to update table state.

Previous group decided to use radiofrequency to communicate between main controller and

table, but the connection is not working properly and some data is lost. The system needs to

assure that all data are arrived in destination. The table used to use NRF24 with controller

which was Arduino Nano. On the specification, both these device must be replaced by a

ESP32 which contains a Wi-Fi chips. All problem occurred during project realization are

developed in this report. On the other hand, Raspberry Pi GPIO is not needed with the Wi-Fi

communication so this device is replaced by a powerful computer running on Linux system.

1. Analysis

3

At the start we had to do couple analysis about this project. The main thing was choosing the

operating- and communication systems. In the following pages are our conclusion and

choices that we are used throughout the project.

1.1. Determining the operating system

We can choose between three common operating systems: Mac OS, Linux or Windows. We

are quickly removed Mac’s option because we did not have this system and it is very

expensive. All members of this group used Windows as main system. However, on the

Raspberry Pi, there is a Linux distribution with specific tools. We have decided to use Ubuntu

Mate with a Virtual Machine because we are using this during courses and it is based on

Debian system like Raspbian which currently installed on the Raspberry Pi.

1.2. System of communication

To create communication between ESPs and computer, we decided to use a server hosted

by the PC. We can make our own protocol. However, we can change the configuration of the

table and thus, change the number of devices connecting on the server. We need to use an

Internet of Things protocol which optimized for this kind of application. In this way, we will use

MQTT protocol because this is included in configuration of MySensor Library which already

used in the current project.

The principle of MQTT protocol is each device can publish or subscribe to a piece of

information on the MQTT server, called broker. Topics can be created to separate the different

information.

For Python we will use paho.mqtt.client library to interact the program with broker. Currently,

Raspberry Pi used serial connection to send a piece of information to NRF24. But we will

remove this so we need to change every serial communication for MQTT communication.

For Arduino, to begin we are using PubSubClient to establish our first communication and to

understand MQTT works. Then, we will move on MySensor Library because this library is

used on current software. Therefore, to minimise code changes on the final program.

MQTT communication system is not familiar to us, so that's why we read about it and learned

how its work. Basically all the information that you need to know about it are in the internet,

with samples how to use it. We already test one version of it, and it was working fine.

2. Technical design

4

2.1. Hardware part

2.1.1. New schematic for one table

For the ESP32, the pin layout is different compared to the Arduino Nano and the NRF24 must

be removed to use Wi-Fi communication instead of radiofrequency. The schematics of one

table has been remade.

We used Fritzing application to draw the wiring. You can find in Figure 1 the schematic of this.

We have taken the previous schematics, cleaned all useless items and connected directly all

wires. After this, we changed Nano and NRF by ESP32.

Instead of Nano, the ESP32 will be supplied by 5V to the USB pin. So RFID-readers are

supplied by ESP32 which control it. For the serial, the transmission pin of boss must be

connected on the receiving pin of helper.

Figure 1. Schematics of the new table

5

2.1.2. Designing PCB

The current table have PCB designed to add components easily without wiring. The problem

was different pin layout and the size of ESP32 compared to Arduino Nano. We decided to

design new version of PCB that setting all modules will be easier. The program that we are

using is Altium Designer.

We started design by drawing the schematic. After that we manufacture PCB at HAN university

of applied sciences and added desired components and pins.

Figure 2. Picture of PCB

2.2. Software part

2.2.1. Transfer Arduino code to ESP32

Changing Arduino Nano to ESP32 and using Wifi communication instead radio frequency

caused changes to program. In this project we used Arduino IDE as a code editor. ESP32 is

not natively used by this IDE. The hardware library must be installed before using it.

The code of Nano must be adapted to running on ESP32. First, many variables must be casted

manually whereas it is automatically casting on Nano compiler. Then, function ESP.restart()

must be used to restart ESP32 instead of asm volatile (" jmp 0"). Next, there are many

warning due to FastLED library. To remove them, #define FASTLED_INTERNAL must be added

before each inclusion of FastLED library. Finally, to remove “the multiple app_main

declaration”, the version 1.0.0 of the ESP compiler must be used.

6

2.2.2. MQTT

First we had to remove NRF24 communication parts, because we don’t need those anymore.

We still left some of it, because the previous communication used also mysensors library as

a base. We set MQTT protocol definitions only to Boss program.

// ---------------To set up table ------------------

// You need change only this file with the following variables:

// Number of flow_segments: 11 for table 1, 12 for others

#define FLOW_SEGMENT_COUNT 11

#define TABLE_ID "1"

#define SSID_WIFI "Tenda"

#define PASSWORD_WIFI "1234567890"

#define MQTT_SERVER_IP 145, 74, 153, 131

#define MQTT_SERVER_PORT 1883 //The default port is 1883

/* ----------- The setup is finished -------------*/

Figure 3. Setting MQTT on Boss programs config.h

Only things you have to do is change a couple things to configure the MQTT. You can see it

from Figure 3.

Here is an explanation about configurations:

SSID_WIFI and PASSWORD_WIFI are your Wifi setting

MQTT_SERVER_IP and MQTT_SERVER_PORT are settings to connect to the broker (see

2.3 to set up this)

TABLE_ID is the number of table. This value can be between 1 and 254 and must be unique

on the smart grid table. For example, on this current configuration, there are six modules from

1 to 6.

#define MY_GATEWAY_MQTT_CLIENT

#define MY_GATEWAY_ESP32

/** Configuration of WiFi */

#define MY_WIFI_SSID SSID_WIFI

#define MY_WIFI_PASSWORD PASSWORD_WIFI

#define MY_HOSTNAME "Boss" TABLE_ID

/** MQTT Configuration **/

#define MY_MQTT_CLIENT_ID "Boss" TABLE_ID

#define MY_MQTT_PUBLISH_TOPIC_PREFIX "sendToPc/" TABLE_ID

#define MY_MQTT_SUBSCRIBE_TOPIC_PREFIX "getFromPc/" TABLE_ID

#define MY_CONTROLLER_IP_ADDRESS MQTT_SERVER_IP

#define MY_PORT MQTT_SERVER_PORT

Figure 4. Ethernet configuration and using the MQTT on programs config.h

7

The Figure 4 shows global variables using by MySensorLibrary to set up MQTT protocol on

ESP32. Under it is Wifi configuration what we did earlier in Figure 3 and after that is MQTT

configuration. There is a particularity with topics because of Serial communication, the first

number is node_id, according to serial API. Unfortunately, with MQTT protocol, this number is

always 0, we have added a new subtopic before: TABLE_ID. However, broadcast cannot be

works with is way, so we are modified Python program to fix it.

2.2.3. Serial communication

Instead of I²C, we used serial communication between one boss and helper. We created a

new code which is easier than the previous one. Indeed we removed ring files and code again

the communication part in loop function for boss part and handle_RFID_message from helper

code.

We initialize a new Serial called Serial2. The baud rate used is the same as the traditional

serial via USB. We keep the default format SERIAL_8N1 which consist in a value of 8 bits,

without parity and one bit stop. The baud rate and the format of message must be the same

for the helper and the boss. Finally, we define transmission pins (cf config.h).

However, the tag_id is a 32 bits message, so we need to split it in 4 messages. We use a loop

for this.

2.2.4. Python code

The program is a portage from the previous project. The main controller is a Raspberry Pi

which is running on Raspbian system with Python 2. We are keeping this configuration

because it is easier to configure the project like this. The new system user MQTT protocol

with a broker which receive connections from all devices. To configure more easily, our

explanations are true for a Debian based-system. For this, we recommend you to install the

last Long-Term Support version of Ubuntu.[2]

8

2.2.4.1. Design phase

The Python program is more complicated than Arduino program. So we took the former UML

graph for the communication between ESPs and Arduino and we changed smart_grid_app.py

and mysensors_gateway_connector.py only.

Figure 5. UML graph for communication in Python

On this schematic, you can see different colors: black for the unchanges, green for the addition

programs and red to deleted or modified programs. Previously, Raspberry Pi used a Serial to

send data to NRF24. Now we use network communication so we can remove Serial library.

There are a lot of devices and the MQTT protocol was created for Internet of Things. It is a

good solution for us, and we implement paho library to use this with Python.

9

2.2.4.2. Implementation phase

The first modifications are in the main file smart_grid_app.py. Indeed, we start with this file to

run the entire program. We needed to change the initialisation of program because we need

the IP address and the listening port of the mqtt broker. When you start the program with

default values, the program will try to connect on local broker with the port 1883. You can use

the following command to start the program:

python2 smart_grid_app.py [IP_address_of_broker] [listening_port_of_broker]

In addition to add arguments in constructor in SmartGridApp object, we change the way to call

GatewayConnector object. Instead of the previous program, we do not need to use thread for

this because paho library already use one.

For GatewayConnector, we redefine all names of variable and the constructor. We create a

function to setup all specifications. We need to add a new function on_connect with

subscription on sendToPc channel and adapt handle_incoming_message prototype to enable

to use paho library.

def create_mqtt_client(self):

#Configure the client mqtt

self.mqtt_client = mqtt.Client()

self.mqtt_client.on_connect = self.on_connect

self.mqtt_client.on_message = self.handle_incoming_message

self.mqtt_client.connect(self.mqtt_ip, self.mqtt_port)

self.mqtt_client.loop_start()

Figure 6. Function to create MQTT client

def on_connect(self, client, userdata, flags, rc):

log('Connected on MQTT broker!'+str(rc))

client.subscribe(self.mqtt_topic_subscribe)

Figure 7. Function to connect on MQTT broker

def handle_incoming_message(self, client, userdata, msg):

Figure 8. New prototype for function for new message incoming

The function with more modification is validate_data() because the previous program just need

to take the message and cut this. Now we use two variables for the message class, “topic”

which contains the root topic and all pieces of information according to this format:

sendToPc/node_id/0/child_id/command/0/type and “message” which contain the

payload. Thus, when we create the message object, we need to remove the topic, the first 0

and add the payload in the end.

10

data_array = str(msg.topic).split('/')[1:]

del(data_array[1])

data_array.append(str(msg.payload))

Figure 9. New code for validate_data function

We make the same kind of modification for send_serial_message function. The name of

function is obsolete but with this way, we do not need to change this in all Python files. After

this, we create an algorithm for broadcast sending: we create a loop which sends in all node_id

from 1 to 254. This program can be improved to send a message to connected table only.

log("Send broadcast:")

log("Topic: " + {0}/255/0/{1}/{2}/0/{3}'.format(self.mqtt_topic_publish,

child_id, command, type) + " and Message: " + str(payload) + "\n")

for table_id in range(1,255):

topic = '{0}/{1}/0/{2}/{3}/0/{4}'.format(

self.mqtt_topic_publish, table_id, child_id, command, type)

self.mqtt_client.publish(topic, str(payload))

Figure 10. Sending broadcast message

11

3. Test report

3.1. Test setup

Test ID Description Expected outcome Actual outcome

Subscription Controllers are subscribed to right topics.

When launching the program both controllers send message that they are subscribed to topics

As expected.

Publishing Controllers are publishing to right topics.

When launching the program both controllers send message that they are subscribed to topics

As expected.

Reading information RFID-readers reads information to table controller.

When reading tag RFID readers reads tag number and pass it to tablecontroller, you can read from serial monitor

As expected.

Connection between boss and helper

Helper- and Boss controller are connected to each other and passing information.

Sending values and information works. When reading tag table connector publish it to topic and main controller reads it. You can see it from serial.

As expected.

Setting up the LEDs LED lights up in right order and color.

Main controller publish value to topic and table controller read it. After that it will set up the LED:s right way.

As expected.

Connection between two modules

Two modules are connected.

RFID-reader on the side of the modules detect new module and set up the new configuration

As expected.

12

3.2. Analysis

As we expected the outcome of these tests went great. When you launch your program both

controllers sends message that they are connected to the right topics and are ready. When

adding module on RFID-reader it sends information that RFID is placed and the ID of the

module to table controller and table controller publish the ID to the right topic using Wifi

connection and MQTT protocol. Then main controller reads the ID from the topic, do

calculations and publish the outcome to another topic. Table controller reads the outcome

from the topic and sets up the LEDs. LEDs are set correctly, the energy is flowing from the

module that produce the energy to the module that consumes it. When placing the module to

the RFID readers which is attached to helper, the ID is sent to the Boss controller by serial

communication and then published to the topic. When removing module from the RFID it will

change the configuration that the energy is not flowing anymore to/from module. If there are

no modules in the table, there is no LEDs on. When adding new module the configuration

automatically changes.

We noticed that the LED setup is going to take some time. The problem is that we are using

virtual computer and it is not the fastest way to communicate. For next project group we

recommend to use computer that uses Linux as operating system.

13

Final conclusions

When we started the project, we have many difficulties to start because Sanni and Milèna do

not know anything about Python, Arduino and Linux and Yoann knows the basics only. On the

other hand, the requests from university and company are very different: whereas our teachers

would want many reports to follow our progress and be assured that we will go on the right

way, our client would have the first result very fast. We did not have any experience on report,

particularly in English, so we were very slow to start the project and documentation. Because

of our difficulties, two Dutch students who works on the other part of this project join us to help

us to progress faster. During this phase, another English student joins our group later, but it

was difficult to work with him because he does not communicate and he leaves one month

after.

After two months, we began to understand the entire project and we make the first results.

Then, we continue like this until the end but sometimes we progress quickly and another day

we fix a very obvious mistake during six hours. However, one month before this end, Milèna

has quit studying at HAN. So, we had to continue this part to finish it and finally, with really

hard work, we reached the project and make all report that school needs with the

documentation for the next group.

To conclude, we are very happy now, at the end of the project. Indeed, we were very afraid at

the beginning because of too many unknown knowledge and problems and we thought we

cannot enable to make this project. But with perseverance, and many searches on the Internet,

despite two persons leave our groups, we have finally done this project. In addition, unlike the

first goal which consists in creating a functional prototype, we are implemented the system on

two tables and the system works completely.