Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
The University of Hong Kong Department of Computer Science
2020/21 COMP4801 Final Year Project
Detailed Project Plan
Body Temperature Measuring System with Smart Patrol Robot
Supervisor Dr. T.W. Chim
Members Lee Ka Fun, Katherine 3035474925
Leung Lok Yi, Harper 3035473359
Tse Man Kit, Jacky 3035477757
Content:
I. Abbreviations p.3
II. List of Figures p.4
1. Introduction p.5
1.1 Current Situation p.5
1.2 Our idea p.5
1.3 Motivation p.6
1.3.1 COVID-19 and fever p.6
1.3.2 Importance of temperature monitoring p.6
1.3.3 Reason on applying mobile thermal camera p.6
2. Objective p.8
3. Methodology p.9
3.1 Equipment set up p.9
3.1.1 Thermal Camera p.9
3.1.2 Normal Camera p.9
3.2 Algorithm and model design p.10
3.2.1 Infrared thermography p.10
3.2.2 Image Segmentation with thresholding methods p.11
3.2.3 Object selection with template matching p.12
4. Risk and challenges p.13
5. Budget p.14
6. Project Management p.15
6.1 Deliverables p.15
6.2 Schedule p.16
6.3 Division of Labour p.18
7. References p.19
I. List of Figures Figure 1. Thermograms of children's faces …………………………………………..……… p. 10
Figure 2. Demonstration of global grey level thresholding ………………….……………… p. 11
Figure 3. Demonstration of object selection for temperature calculation ……....…………… p. 12
II. Abbreviations
COVID-19 Coronavirus Disease 2019
HKAA Hong Kong Airport Authority
HKIA Hong Kong International Airport
IR Infrared radiations
WHO World Health Organization
1. Introduction
1.1 Current Situation In light of COVID-19, HKIA has adopted a series of preventive measures to safeguard public
health, including disinfection channels and autonomous sterilisation robots [1]. In addition,
arriving, departing and transit passengers are required to have their body temperature checked for
fever [2]. Despite strict measures of health screening being implemented in airports around the
globe[3], there are reported cases of passengers cheating thermal screening by taking fever
reducers[4][5]. To combat this situation, we aspire to utilise the use of the smart patrol robot to
enhance the future health measure in HKIA.
1.2 Our idea To ensure the health condition of every passenger before onboarding, the smart patrol robot can
be the second line of defense in the airport. After going through the health screening, passengers’
body temperatures will be monitored by the smart patrol robot that is keeping watch in the
boarding gate areas. When a suspected fever case is spotted, using the existing alert mechanism,
the robot will alert health control staff to handle the case by delivering images of the fevering
passengers and their location details.
1.3 Motivation
1.3.1 COVID-19 and fever
COVID-19 is an infectious disease originated from Wuhan, China, in December 2019 [6]. As of
26th of September, 2020, there have been at least 32.6 millions confirmed global cases of
COVID-19, causing more than 990,000 global deaths [7]. Declared to be a global pandemic by
the WHO [6], the spread of COVID-19 brings challenges to humanity. A study in June 2020
confirmed fever as the most prevalent symptoms among confirmed COVID-19 cases, where 78%
of patients reported fever [8]. Since air travel plays a vital role in the spread of COVID-19 [9],
speedy identification of airport users who are having a fever may help screen COVID-19
patients, which may in turn benefit the containment of the pandemic.
1.3.2 Importance of temperature monitoring
Besides COVID-19, fever is the major symptoms of other communicable diseases such as Ebola
and hand-foot-and-mouth disease. If those showing signs of a fever perform self quarantine and
refrain from having human interactions, control of such pandemic can become easier. However,
not all patients are aware of their fever, and some are reluctant to report themselves to the
authorities [10]. Therefore, it is more accurate and more reliable, to perform constant
temperature monitoring, instead of entrusting self-reporting.
1.3.3 Reason on adding mobile thermal camera
HKIA is currently using stationary thermal cameras for checking the body temperature of
travellers. Fixed cameras are set and airport uses are instructed to walk past the cameras, so that
their temperature can be assessed.
Since there is usually a large number of travellers walking past the stationary cameras at a time,
measurement may be obstructed due to blocking of the camera view. When mobile thermal
cameras are added for assistance, thermal images of airport users can be captured from different
angles. Also, since the robot is patrolling to different locations of the airport, camera blindspots
are avoided. Therefore, adding mobile thermal cameras to the existing stationary ones can
provide the airport with better monitor coverage. This can act as the second line of defense
against fevering passengers
2. Objective The project aims to build a mobile temperature measuring instrument for HKIA with the smart
patrol robot. The objectives of this project are listed below.
1. Incorporate a thermal camera in the robot
2. Develop a thermal image segmentation algorithm using different thresholding methods
3. Compare the result of using different thresholding methods for specific range of human
temperature
4. Develop a object selection algorithm for human faces in processed thermal images
5. Develop an algorithm for calculating the human temperatures from processed thermal
images
3. Methodology
3.1 Equipment set up
3.1.1 Thermal Camera
Thermal Camera converts heat energy released into visible light such that we can collect the data
to make analysis on the temperature of that object. When a thermal camera is used to check
humans, we are able to collect the body temperature of travellers.
By using thermography, the image collected from the camera, also known as thermogram, will
then be analyzed. Moreover, the image collected can be projected on the screen for further
interpretation.
3.1.2 Normal Camera
Other than the thermal camera, a normal camera will also be installed. Due to the fact that
thermal cameras can only produce thermal images as shown in Figure 1, it would be difficult for
the staff to identify the facial features of travellers. With the assistance of a normal camera, we
are able to capture the real appearance of the targeted passengers. The normal images combined
with the thermal images can help airport staff spot the fevering passenger more easily.
3.2 Algorithm and model design
3.2.1 Infrared thermography
According to the black body radiation law, all objects of temperature above absolute zero emits
infrared radiation [10]. The amount of radiation emitted by an object increases with its
temperature. Infrared thermography allows the temperature of human bodies to be captured in
the form of a thermogram (Figure 1), where the surface temperature of living things can be
monitored [11]. Since there is great correlation between thermal physiology and skin
temperature, in the medical field, infrared thermography is used to diagnose breast cancer,
diabetes neuropathy and peripheral vascular disorders, etc. [12]. With COVID-19 going on a
rampage, we are particularly interested in its capability to detect fever.
Figure 1 [13]. Thermograms of children's faces. The left shows normal body temperature, while
the right shows raised temperature around the eyes and nose, indicating a possible fever.
3.2.2 Image Segmentation with thresholding methods
Image segmentation is the process of partitioning an image into different parts with respect to
specific qualities, for instance, textures and pixel gray level[14]. Global grey level thresholding
is one of the most popular thresholding methods for image segmentation [15]. It is used to
differentiate between an object from its background by setting a certain grey level as the
threshold[16]. For example, if the grey level of a pixel goes above the threshold, it is classified as
the object and is changed to a certain colour, such as, white. If the grey level of a pixel goes
below the threshold, it is classified as the object and is changed to another colour, such as, black.
Figure 2 [17]. Demonstration of global grey level thresholding. The left shows an unpressed
image with the blue area as the background and the green area as the object. The middle shows
the histogram with x-axis showing the grey level and the y-axis showing the number of pixels. It
indicates that the background and the object can be differentiated if the grey level threshold is
set at 0.5. The right shows the result of global grey level thresholding.
This thresholding method can be leveraged to process thermal images from the footage recorded
by the thermal camera and obtain the size and positions of human bodies in the images.
3.2.3 Object selection with template matching
After image segmentation, images of human bodies can be differentiated from the background.
As we focus on measuring the temperature of the facial area, further object selection is needed to
sort out facial regions from the image. Template matching can be used to extract the facial area
by choosing elliptical and circular templates whose shapes are similar to typical face contour
[18].
(Figure 3a) Thermal image before processing
(Figure 3b) Thermal image after grey level thresholding
(Figure 3c) Thermal image after facial area is identified
Figure 3 [19]. Demonstration of object selection for temperature calculation.
As the facial area can be located, the average temperature can be calculated with reference to the
original thermal image.
4. Risk and challenges
Description Consequences Mitigation
Practical constraint:
Access control at the HKIA allows
only those taking a flight to enter
the airport
Testing in the airport may be
difficult
Seek assistance from HKAA so
that we can be allowed in the
airport under supervision.
Technical constraint:
Both the camera and measuring
subjects are moving constantly
It may be challenging to
capture the infrared radiation
from the subjects
Use software means (e.g.
programs/ scripts) to perform
adjustments and calibration.
Technical constraint:
There is a very thin margin between
normal body temperature and fever
(for example, forehead temperature
of 35.9°C is considered normal but
36.0°C may be considered as an
indication of fever)
False positive cases, or false
negative cases.
We need to calibrate our
functioning mechanism with the
thermal camera, such as
readjusting our cut-off
temperature, performing multiple
measurements and taking the
average, etc..
5. Budget
Item Description Quantity Price
Meridian Innovation -
MI0801 Camera Module
Evaluation Kit
The kits will be installed to the patrol
robot and used to capture thermal
images
2 ~$1400HKD
Travel expense Travel cost for travelling to airport for
research and data collection
1 $600HKD
Total: $2000HKD
As our project is supported by the Teaching Development and Language Enhancement Grant
(TDLEG), we were offered a maximum grant of HK$8,000 in the form of reimbursement upon
completion of the project. We are confident that our budget can be fully covered by the grant.
6. Project Management
6.1 Deliverables
Date Deliverables
4 October, 2020 Phase 1 (Inception) : • Detailed project plan • Project web page
11-15 January 2021 First presentation
- Progress updates
24 January, 2021 Phase 2 (Elaboration): • Preliminary implementation
- Purchase of cameras - Installation of thermal camera - Installation of normal camera - Implementation of the human face identification function
• Detailed interim report
18 April, 2021 Phase 3 (Construction):
• Finalized tested implementation
- Implementation of fever detection function - Development of the alerting mechanism (when a fever
case is discovered)
• Final report
19-23 April, 2021 Final presentation
4 May, 2021 Project exhibition
6.2 Schedule
Date Milestones
September 2020 Deliverables of Phrase 1: ● Detailed Project Plan ● Project Web Page
Review:
● Manual of the smart patrol robot provided by HKAA Research:
● Existing features and capabilities of the smart patrol robot ● Thermal camera for the project
October 2020 Research: ● Algorithms on image thresholding and object selection
Implementation:
● Basic navigation of the smart patrol robot ● Installation of thermal camera
November 2020 Research: ● Algorithms on image thresholding and object selection
Implementation:
● Data collection and processing ● Preliminary implementation of algorithms of image thresholding
Reviews:
● Evaluation of the result of different algorithms of image thresholding
December 2020 Preparation of Phrase 2: ● Interim report ● First presentation ● Product Demonstration in the first presentation
Implementation:
● The best performing image thresholding algorithms
● Preliminary implementation of algorithms of image thresholding ● Developing algorithms for object selections
January 2021 Deliverables of Phrase 2: ● Interim report ● First presentation ● Demo in the first presentation
Implementation:
● Evaluation of object selections algorithms ● Developing body temperature measuring algorithms
Research:
● Existing APIs of the robot to send out information
February 2021 Implementation: ● Testing of temperature measure algorithms ● Fine tune the image thresholding and object selection algorithms ● Information delivery framework of the robot
March 2021 Preparations of Phrase 3: Final report
● Final presentation ● Product Demonstration the final presentation ● Integration test and debug the whole project
April 2021 Deliverables of Phrase 3: ● Final report ● Final presentation ● Product Demonstration the final presentation ● [if selected] Preparation for the project competition
6.3 Division of Labour
Task Lee Ka Fun Leung Lok Yi Tse Man Kit Estimated Time spent
(days)
Beginning and Basic Research
Detailed Project Plan ✓ ✓ ✓ 10
Project Webpage ✓ 3
Review on technical document about the smart patrol robot provided by HKAA
✓ ✓ ✓ 3
Research on existing features of the smart patrol robot
✓ ✓ ✓ 3
Research on existing algorithm adopted in the smart patrol robot
✓ ✓ ✓ 3
Time Spent 22
Elaboration
Installation of thermal camera ✓ 2
Installation of normal camera ✓ 2
Research on algorithms on image thresholding and object selection
✓ 5
Design temperature measure algorithms ✓ ✓ ✓ 7
Implementation of temperature measure algorithms
✓ 10
Testing of temperature measure algorithms
✓ ✓ 5
Design image thresholding and object selection algorithms
✓ ✓ ✓ 7
Implementation of image thresholding and object selection algorithms
✓ ✓ 10
Testing of image thresholding and object selection algorithms
✓ ✓ 5
Preliminary implementation of algorithms of image thresholding
✓ ✓ ✓ 10
Design object selections algorithms ✓ ✓ ✓ 7
Implementation of object selections algorithms
✓ ✓ 10
Testing of Object selections algorithms ✓ ✓ ✓ 5
Interim report ✓ ✓ ✓ 10
First Presentation ✓ ✓ ✓ 10
Time Spent 95
Finalization
Integration test and debug the whole project
✓ ✓ ✓ 10
7. References [1] "HKIA Applies Advanced Technology to Step Up Disinfection Against COVID-19", Hong
Kong International Airport, 2020. [Online]. Available:
https://www.hongkongairport.com/en/media-centre/press-release/2020/pr_1446. [Accessed: 30-
Sep- 2020].
[2] "Department of Health - Health Control Measures for travellers at airport, seaport and land
boundary control points", Info.gov.hk, 2020. [Online]. Available:
https://www.info.gov.hk/info/sars/en/boundarycontrol.htm. [Accessed: 30- Sep- 2020].
[3] "Coronavirus: airports around the world carry out screenings", the Guardian, 2020. [Online].
Available:
https://www.theguardian.com/science/2020/jan/21/coronavirus-screenings-global-travelling-airp
ort. [Accessed: 30- Sep- 2020].
[4] "Passengers cheat flu scan with fever reducers", Abc.net.au, 2020. [Online]. Available:
https://www.abc.net.au/news/2009-06-15/passengers-cheat-flu-scan-with-fever-reducers/171426
4. [Accessed: 30- Sep- 2020].
[5] "A Chinese embassy in Paris tracked down a woman who gloated on social media about
cheating airport detection with a medicine that lowered her fever", Business Insider, 2020.
[Online]. Available:
https://www.businessinsider.com/wuhan-coronavirus-woman-avoided-airport-tests-travel-france-
2020-1. [Accessed: 30- Sep- 2020].
[6] Q&A on coronaviruses (COVID-19). (n.d.). World Health Organization. [Online].
Available:
https://www.who.int/emergencies/diseases/novel-coronavirus-2019/question-and-answers-hub/q-
a-detail/q-a-coronaviruses#:~:text=symptoms.
[Accessed Sep. 27, 2020].
[7] COVID-19 Dashboard. (n.d.). Center for Systems Science and Engineering at Johns Hopkins
University. [Online]. Available:
https://gisanddata.maps.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467
b48e9ecf6. [Accessed Sep. 27, 2020].
[8] M. Grant, L. Geoghegan, M. Arbyn, Z. Mohammed, L. McGuinness, E. Clarke, R. Wade.
(2020, June 23). “The prevalence of symptoms in 24,410 adults infected by the novel coronavirus
(SARS-CoV-2; COVID-19): A systematic review and meta-analysis of 148 studies from 9
countries”. [Online]. Available:
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0234765. [Accessed Sep. 27,
2020].
[9] H. Nakamuraa, S. Managi, “Airport risk of importation and exportation of the COVID-19
pandemic”, Transport Policy, vol. 96, pp. 40-47, Sep 2020.
[10] S. Stewart, R. Johnson. Blackbody Radiation: A History of Thermal Radiation
Computational Aids and Numerical Methods. Cleveland, Ohio: CRC Press, 2016.
[11] G. Gaussorgues, S. Chomet, Infrared Thermography, London, U.K.: Chapman and Hall,
1994.
[12] B. Lahiri, S. Bagavathiappan, T. Jayakumar, J. Philip. “Medical applications of infrared
thermography: A review”, Infrared Physics & Technology, vol. 55, pp. 221-235, July 2012.
[13] F. Ring, “The Herschel Heritage to Medical Thermography”, Journal of Imaging, vol. 2, pp.
13 April 2016.
[14] Zheng, Xiulian, et al. “Image Thresholding Based on Gray Level-Fuzzy Local Entropy
Histogram.” IEEJ Transactions on Electrical and Electronic Engineering, vol. 13, no. 4, 2018,
pp. 627–631.
[15] S. Sarkar and S. Das, "Multilevel Image Thresholding Based on 2D Histogram and
Maximum Tsallis Entropy— A Differential Evolution Approach," in IEEE Transactions on
Image Processing, vol. 22, no. 12, pp. 4788-4797, Dec. 2013, doi: 10.1109/TIP.2013.2277832.
[16] Parker, J.R. “Gray Level Thresholding in Badly Illuminated Images.” IEEE Transactions on
Pattern Analysis and Machine Intelligence, vol. 13, no. 8, 1991, pp. 813–819.
[17] "2.6.8.16. Histogram segmentation — Scipy lecture notes", Scipy-lectures.org, 2020.
[Online]. Available:
http://scipy-lectures.org/advanced/image_processing/auto_examples/plot_histo_segmentation.ht
ml. [Accessed: 30- Sep- 2020].
[18] H. F. Haghmohammadi, D. S. Necsulescu and M. Vahidi, "Remote measurement of body
temperature for an indoor moving crowd," 2018 IEEE International Conference on Automation,
Quality and Testing, Robotics (AQTR), Cluj-Napoca, 2018, pp. 1-6, doi:
10.1109/AQTR.2018.8402698.
[19] "A tutorial on Camera based Thermal Screening using Computer Vision", Medium, 2020.
[Online]. Available:
https://medium.com/@feat7/a-tutorial-on-camera-based-thermal-screening-using-computer-visio
n-d8b3304c08e3. [Accessed: 30- Sep- 2020].