Vision-based Human Detection System for Internet of ThingsTeam Leader: Ryan ReynoldsTeam Members: Brett Dawson, Mina Kamel, Botros ShenoudaFaculty Advisor: Dr. Mohammad S. ShiraziElectrical Engineering and Computer Science, Cleveland State University, Cleveland, OH 44108

Real-time human detection and tracking is a vast, challenging and important field of

research. It has wide range of applications in human recognition, human computer

interaction, and video surveillance. This project aims to develop a low-cost system

capable of public surveillance. The software identifies and counts the number people in

a public space using a monocular camera. The detections are then transferred to the

cloud for further analysis. The software is implemented on a Raspberry Pi 3B+ single

board computer. The hardware operating expenses total $219 with the Intel Neural

Compute Stick 2 or $139 without. Three exclusive phases have been conducted in the

span of nine months to insure the highest results presented to launch the project: initial

research of various detection algorithms and implementation on a Raspberry Pi,

development of detection algorithms, and optimizing these algorithms on the Raspberry

to obtain human detection at 11.8 fps.

Abstract

• Advancements in hardware has caused a resurgence in popularity of Convolutional

neural networks (CNNs) for computer vision applications.

• These techniques can be used to solve problems that humans are incapable of

solving, such as mass surveillance and real time traffic control.

• The concurrent rise of IoT and cloud computing has created a demand for computer

vision detectors to be run on the edge.

• The cost of these detectors has been a barrier of entry for scaling accurate systems.

• The project aims to evaluate the state of detection systems on the edge by

implementing human detection on a low cost system with traditional and newer

techniques.

• The project also provides a sample use case for the human detection system.

Introduction and Background

r.reynolds27@vikes.csuohio.edu

System Design and Algorithms

Experimental Results

Figure 2 : YoloV3 and TinyYoloV3 Visualization

System Overview• Implement human detection using OpenCV on Raspberry PI 3B+ and detection methods:

○ Evaluation of traditional detection methods:

○ Haar & Hog

○ Evaluation of state of the art detection methods (Convolutional Neural Network)

○ YoloV3 & TinyYoloV3

• Upload detections to Azure IoT Hub route to different endpoints.

• NodeJS web server is a sample endpoint that post processes data.

• Semi-static HTML page displaying post processed data to the clients on a map and in

graphical form.

Figure 3: YoloV3 CNN Diagram

▪ Algorithms initially implemented in Python

▪ Python was too slow (Interpretation vs. Compilation)

• OpenCV detection libraries written in C but wrapped for Python.

▪ Reimplemented each algorithm in C++

• more efficient and faster than python (still not real time)

▪ Added Intel Neural Compute Stick 2 (NCS2)

• achieved real time detection with acceptable accuracy via TinyYoloV3 (MAP

30% 11.8 fps).

• High accuracy non-real time alternative via YoloV3 (MAP 50% 1.79 fps).

▪ Created a sample traffic controller endpoint displaying runtime and # of humans

in real time on a map location and graphically.

● Use TinyYoloV3 for real time human detection in C++ with the NCS2.

● Use YoloV3 C++ with the NCS2 for increased accuracy non-real time applications.

● Raspberry Pi 3B+ is not ready for modern real time computer vision on its own.

● However, our map endpoint shows detectors running fps<10 & fps>1 can have practical use

cases.

Conclusion and Future Recommendations

27

Figure 4: HAAR Features Figure 5: HOG Features

Figure 7: Raspberry Pi Camera Feed TinyYoloV3

• Implement Tracking

• Evaluate Mobilenet SSD

• Change single board computer

to Nvidia Jetson Nano

Recommendations• Increase YoloV3 performance

• Add automation for starting detection from client side

• Integrate multiple detection systems

• Add other objects besides humans

Figure 8: Map Endpoint

Table 1: FPS and CPU Load for each algorithm

Figure 1: System Model

Figure 6: Average precision per algorithm

Haar low cost and low accuracy Hog medium cost medium accuracy

Algorithm Info● YoloV3 high cost and high accuracy

● TinyYoloV3 medium cost and medium accuracy