Embed Size (px)
Citation preview
AUTONOMOUS NAVIGATION IN INDOOR MAPPED
AND UNMAPPED ENVIRONMENTS AS AN
ASSISTIVE TECHNOLOGY
____________________________________
A Thesis
Presented to the
Faculty of
California State University, Fullerton
____________________________________
In Partial Fulfillment
of the Requirements for the Degree
Master of Science
in
Computer Engineering
____________________________________
By
Harkishan Singh Grewal
Thesis Committee Approval:
Kiran George, Computer Engineering Program, Chair
Kenneth John Faller II, Computer Engineering Program
Pradeep Nair, Computer Engineering Program
Spring, 2018
ii
ABSTRACT
Autonomous navigation has risen to prominence in public discourse in recent
years. Pursuits of a self-driving car by notable tech companies, Tesla and Waymo
(formerly Google self-driving car project), have brought autonomous navigation to the
forefront and inspired much excitement. However, the applications of autonomous
navigation are not limited to self-driving cars. A potential and promising application is to
modify electric wheelchairs to improve the lives of individuals with disabilities.
Individuals with motor neurodegenerative diseases and spinal cord injuries cannot
use conventional wheelchairs, leaving them immobile and stripped of autonomy. An
autonomous wheelchair capable of navigation in both mapped and unmapped indoor
environments is proposed to restore mobility and autonomy to such individuals. The
wheelchair uses various sensor to detect stationary and moving obstacles in real-time,
enabling it to safely chart a course to its destination. Several sets of trials are conducted
to test the functionally of the proposed system in different mapped and unmapped
environments. The proposed system successfully and autonomously navigated to its
destination in most trails with no collisions. These results are promising and validate the
design, usability, and safety of the proposed system. They show potential of enriching the
lives of individuals with disabilities with autonomy and mobility.
iii
TABLE OF CONTENTS
ABSTRACT ................................................................................................................... ii
LIST OF TABLES ......................................................................................................... v
LIST OF FIGURES ....................................................................................................... vi
ACKNOWLEDGMENTS ............................................................................................. viii
Chapter
1. INTRODUCTION ................................................................................................ 1
2. RELATED WORK ............................................................................................... 3
3. ROBOT OPERSTING SYSTEM ......................................................................... 5
Packages................................................................................................................ 6
gmapping ....................................................................................................... 6
map_server ..................................................................................................... 6
AMCL ............................................................................................................ 6
Move_base ..................................................................................................... 7
Nodes .................................................................................................................... 8
Master ................................................................................................................... 8
Topics.................................................................................................................... 9
4. TOOLS AND METHODS ................................................................................... 10
Autonomous Navigation System .......................................................................... 10
Sensor System ....................................................................................................... 11
LIDAR ........................................................................................................... 12
Proximity ....................................................................................................... 13
Odometry System ................................................................................................. 17
Wheelchair Control System .................................................................................. 18
User-Input Device ................................................................................................. 19
User-Interface ....................................................................................................... 20
iv
5. EXPERMINTAL DESIGN, RESULTS, AND DISCUSSION ............................ 23
Mapped Navigation............................................................................................... 23
Case 1 ............................................................................................................. 23
Case 2 ............................................................................................................. 27
User-Interface ....................................................................................................... 29
Evaluation Metrics ......................................................................................... 29
Results ............................................................................................................ 30
Discussion ...................................................................................................... 30
Unmapped Navigation .......................................................................................... 31
Evaluation Metrics ......................................................................................... 34
Results ............................................................................................................ 35
Discussion ...................................................................................................... 40
6. CONCLUSION ..................................................................................................... 41
REFERENCES .............................................................................................................. 43
v
LIST OF TABLES
Table Page
1. Key Specification for RPLIDAR A1 and A2 from Slamtec ................................ 12
2. Results for Case 1 ................................................................................................ 25
3. Results for [8]’s Task 2 ........................................................................................ 26
4. Results for Case 2 ................................................................................................ 28
5. Results for ANS with Sip-and-Puff Device and User-Interface .......................... 30
6. Results for User-Interface .................................................................................... 30
7. Results for Unmapped Navigation to First Intermediary Destination ................. 39
8. Results for Unmapped Navigation to Second Intermediary Destination ............. 40
9. Results for Unmapped Navigation to Final Destination ...................................... 40
vi
LIST OF FIGURES
Figure Page
1. Block diagram of ANS......................................................................................... 11
2. Autonomous wheelchair with a LIDAR, rotary encoders, sip-and-puff device,
and user-interface ................................................................................................. 14
3. A test case scenario where the autonomous wheelchair is obstructed by
a chair and bookcase ............................................................................................ 15
4. Local costmap generated using LIDAR data alone. The bookcase is detected
as an obstacle, but the chair is not ....................................................................... 16
5. Local costmap generated using LIDAR and proximity sensor data. Both the
bookcase and chair are detected as obstacles ....................................................... 16
6. Breeze Sip-and-Puff device from [29] ................................................................. 20
7. Main menu of user-interface ................................................................................ 21
8. Navigation screen of user-interface ..................................................................... 22
9. Map of environment used in [8]’s Task 2 ............................................................ 24
10. Generated map of environment used for Case 1 showing distances and
approximate path of travel in trials ...................................................................... 24
11. Mapped environment with measurement and approximate path for Case 2 ........ 27
12. First destination for shopping mall test case ........................................................ 32
13. Second destination for shopping mall test case ................................................... 33
14. Third destination for shopping mall test case ...................................................... 34
15. Initial view of local costmap for unmapped navigation....................................... 36
16. Local costmap showing first intermediary destination in range of wheelchair ... 37
vii
17. Local costmap showing second intermediary destination in range of
wheelchair ............................................................................................................ 38
18. Local costmap showing final destination in range of wheelchair ........................ 39
viii
ACKNOWLEDGMENTS
First and foremost, I acknowledge my family and friends for their continuous
support over the years in pursuit of higher education. Their encouragement and advice
allowed me to push forward in my endeavors.
I would like to express my gratitude to Dr. Kiran George for his guidance,
support, and valued advice. I especially thank him for giving me a research assistant
position within the Bio-Electric Signal Based Systems Laboratory, enabling me to
undertake the presented research. I acknowledge the faculty and staff of the Computer
Engineering Program for their wealth of knowledge and support.
Lastly, I thank my colleagues within the lab and the Integrated BS-MS Degree
Program. Their friendship and advice are dear to me and guided me through the many
challenges. I cherish them for their companionship, humor, and wit.
1
CHAPTER 1
INTRODUCTION
2.2 million people in the United States use wheelchairs for daily tasks and
mobility according to the National Institute of Child Health and Human Development
(NICHD) [25]. 300,000 people in the US have spinal cord injury (SPI) and there are
11,000 more SPIs every year. Over 40% of SPI patients are quadriplegic with no control
over their limbs [26] and require wheelchairs for mobility.
Since their advent in the early 1900s, electric wheelchairs have helped individuals
regain their lost mobility from injuries. Most conventional electric wheelchairs have a
joystick mounted on an armrest which is used to control the motion of the wheelchair [1].
However, individuals suffering from neurodegenerative diseases such as amyotrophic
lateral sclerosis (ALS), sometimes called Lou Gehrig’s disease, cannot operate
conventional electric wheelchairs due to limited mobility in their limbs. ALS is a rapidly
progressing neurological disease which attacks motor neurons responsible for controlling
muscles for movement [2]. Individuals with ALS lose fine motor control in their limbs as
the disease progresses leaving them unable to operate conventional joystick-controlled
electric wheelchairs. As a result, ALS patients are immobile for prolonged periods of
time or heavily dependent on others for mobility.
The elderly experience a decline in motor function due to the aging process,
limiting their autonomy and mobility. Populations in most developed countries are aging
2
and life expectancy is increasing in developing countries with better access to healthcare
and nutrition [5]. As the world population ages, there will be increasing demand for
solutions to address the issue of declining mobility.
Limited mobility can significantly decrease motivation [3]. Loss of autonomy and
mobility drastically decreases the quality of life of a person [4]. Furthermore, limited
mobility generates frustration and anxiety and can cause depression [3].
In the presented work, a wheelchair capable of autonomous navigation in mapped
and unmapped indoor environments is proposed to address the issues of limited mobility
and reduced autonomy. A wheelchair is considered autonomous when it can navigate to a
given location in an unstructured environment without continuous human guidance [4].
The proposed wheelchair can navigate itself to a desired location in both mapped and
unmapped indoor environments, all while detecting and avoiding obstacles in real-time.
Several other solutions that have been proposed over the years to address the
same issues are explored in Chapter 2. A brief overview of the Robot Operating System
(ROS), which provides the framework for the development of the proposed wheelchair, is
provided in Chapter 3. Chapter 4 presents the various tools and methods used to develop
the autonomous wheelchair. Chapter 5 presents experiments used to validate the
performance and capabilities of the wheelchair. The results of the experiments are
presented and discussed in Chapter 5 as well. Chapter 6 discusses the significance of the
presented work and key conclusions to be drawn from the work.
3
CHAPTER 2
RELATED WORK
There has been interest in developing smart wheelchairs to alleviate the problem
of limited mobility for many years. Several wheelchairs have been developed which use
eye blinks, eye movements, facial/head gestures, and brain waves for control [27].
In 1999, Levine et al. [28] introduced the NavChair Assistive Wheelchair
Navigation System which sought to reduce the cognitive and physical requirements of
operating a conventional electric wheelchair. The NavChair used a DOS-based computer
system and ultrasonic sensors to avoid obstacles, navigate doorways, and follow walls.
J. S. Nguyen et al. [3] proposed the Thought-controlled Intelligent Machine (TIM)
with a unique camera configuration for 3D depth perception mapping and a spherical
camera system for 360-degrees of monocular vision. TIM implements two hands-free
technologies: Head-movement controller (HMC) and brain-computer interface (BCI).
HMC enables a user to navigate and control TIM using their head movements. BCI
measures a user’s neurological activity which is used by TIM to navigate.
Wee-Kiat et al. proposed a steady state visual evoked potential (SSVEP) based
BCI wheelchair to navigate [6]. The wheelchair focused on transferring control between
the user and navigation system to reduce the number of decisions made by the system.
Kuo et al. proposed an omni-wheeled mechanical platform design for an
autonomous wheelchair [7]. The design primarily focused on addressing the problem of
4
navigating in a confined environment frequently found in hospitals, homes, libraries, etc.
The design minimized the size of the wheelchair and reduced its power consumption.
Iturrate et al. proposed a noninvasive brain-actuated wheelchair that relies on a
P300 neurophysiological protocol and automated navigation [8]. The wheelchair presents
a virtual reconstruction of the environment allowing the user to select a location using
EEG. The autonomous navigation system receives the location and navigates while
avoiding obstacles.
These wheelchairs are limited by the high cost of specialized hardware and the
strain placed on the user during navigation. Recent developments in automation have
been incorporated into wheelchairs [8] to automate navigation to reduce strain on the
user.
While autonomous navigation in outdoor environments has been extensively
explored, indoor environments present a challenge for autonomous navigation due to
unavailability of GPS signals for localization. WiFi signals, Bluetooth modules, and
Radio Frequency transmitter/receivers are commonly substituted for GPS and used to
localize a wheelchair during navigation [31] - [34]. This approach requires a robust
infrastructure to localize properly and cannot be scaled to large indoor environments, i.e.
shopping malls, airports, etc.
5
CHAPTER 3
ROBOT OPERATING SYSTEM
The robot operating system (ROS) is an open source platform that runs on top of
Unix-based platforms and provides a flexible framework for writing robot software [10,
14]. ROS provides several services expected from an operating system, including
hardware abstraction, low-level device control, inter-process communication, and
package management [14]. However, ROS is not a stand-alone operating system and
requires a Unix-based platform to fulfill any deficiencies.
More importantly, ROS serves as a collaborative software development
mechanism to accelerate robotic development. It aggregates tools, libraries, and
conventions developed by various laboratories, organizations, and developers to simplify
the tasks of developing robust robotic systems. Robotic systems face a series of complex
challenges from localization to path-planning to obstacle detection. These challenges are
further confounded by variations in environments making the development process
difficult and lengthy for any individual or institution. ROS enables individuals to
specialize in specific aspects of robotic development and share their expertise enabling
others to develop systems quickly and with greater ease.
ROS is a modular system allowing aspects of a robotic system to be controlled at
fine-granularity. Several keys aspects of ROS include packages, nodes, master, and topics
[15].
6
Packages
Packages are used within ROS to organize software and maintain modular design.
Packages are used to create modules with specific tasks and allows for software to be
reused easily. ROS packages usually contain ROS nodes, datasets, configuration files, or
anything else that be useful for the module [16]. Several key packages for the
development of an autonomous wheelchair are gmapping, map_server, AMCL, and
move_base.
Gmapping
Gmapping is an implementation of the laser-based Simultaneous Localization and
Mapping (SLAM) algorithm for ROS [17]. It uses laser scan data about the environment
and obstacles to create a 2D occupancy grid map. The generated map shows both free
space (no obstructions detected) and obstacles if the scan reflects before reaching its
maximum range of 6 meters. The generated maps can be used by the wheelchair for
navigation and path-planning.
Map_server
Map_server package allows maps generated by the gmapping package to be saved
for later use. This package also allows previously generated and saved maps to be
reloaded and used for navigation.
AMCL
AMCL package serves as a probabilistic localization system for a robot
navigating a 2D environment [18]. It implements the adaptive Monte Carlo localization
(AMCL) approach described by D. Fox et al. [12]. AMCL makes use of a particle filter to
track the pose of a robot against a known map. AMCL takes a laser map generated by
7
gmapping package along with laser scans from a LIDAR and transform messages as
inputs. It uses the particles filter with specified parameters to output pose estimates for
the wheelchair. These pose estimates allow the ROS to localize the wheelchair within a
saved map and enable path-planning to initiate.
Move_base
The move_base package enables the wheelchair to travel to a given goal [19]. It
accomplishes the navigation task by linking together global and local costmaps and
planners. Costmaps continuously receive inputs from sensors allowing the positional of
the obstacles to be updated dynamically. The local costmap is generally smaller than the
global costmap and primarily handles obstacles very close the wheelchair. The global
costmap is much larger than the local costmap and takes a map as an input in addition to
data from sensors.
The planners work with their associated costmaps to generate path plans to reach
navigation goal. The global planner generates a global plan based on the input static map
using Dijkstra’s algorithm [11]. The path-planning algorithm is modified to compute
Euclidean distance estimations. The local planner takes the global plan and local costmap
data as inputs. It uses a greedy algorithm to generate its own plan based on the global
plan and the positions of obstacles on the local costmap. As a result, the local plan is
often more efficient and faster, but increases the probability of a collision. This is offset
by the local planner’s ability to incorporate new and dynamic obstacles that may be
emerge after the generating of static map. The local planner produces a series of short-
term goals to achieve the navigation goal. The trajectory of the local plan is controlled by
8
a dynamic window approach (DWA) algorithm [12]. It functions by assigning point
values to the space to be traversed and scoring those points based on optimal path.
Nodes
ROS nodes are processes that perform specific computations [20]. Several nodes
are collected together in a graph to accomplish tasks. Nodes can communicate with one
another in the ROS paradigm using channels called ROS topics. Nodes provide several
advantages to the overall system. Nodes add fault tolerance by isolating issues to
individual nodes. This means that a crash in one subsystem does not crash the whole
system making the whole system more robust. Nodes also aid in debugging by isolating
issues. Nodes serve to reduce code complexity. Nodes allow tasks to be broken down and
implemented in individual nodes. Nodes abstract away implementation details and
provide a simple application programming interface (API). This simplifies the task of
configuring the node in the rest of the graph.
Master
The ROS master is a specialized node that provides naming and registration
services to the other nodes in the ROS system [21]. It keeps track of all nodes and topics
when they are created. The ROS master enables ROS nodes to locate one another and
request for a peer-to-peer communication over a ROS topic. A trivial case to demonstrate
the role of the ROS master is presented below where node1 wishes to send data to node2
over topic1:
Step 1: Node1 informs ROS master that it will be publishing data to a ROS topic
called “topic1”
Step 2: ROS master created a topic called “topic1” and adds Node1 as publisher
Step 3: Node2 requests to receive data from topic1
9
Step 4: ROS master adds Node2 as a subscriber to topic1
Topics
ROS topics are communications channels for ROS nodes to communicate with
one another [22]. ROS topics provide anonymous communication, decoupling the
production of information from its consumption. This means nodes that publish data on a
given topic are not aware of nodes that subscribe to data on that topic. As such, ROS
topics can have support multiple publishers and subscribers.
10
CHAPTER 4
TOOLS AND METHODS
Several keys systems and platforms are required for autonomous navigation of a
wheelchair. Notable among these are the autonomous navigation system (ANS), sensor
system, odometry system, and wheelchair control systems. Each component is described
in detail in this chapter.
Autonomous Navigation System
The ANS is built primarily upon the move_base package within ROS. It inputs
from the sensor system about the environment and obstacles. This information is passed
the global and local costmaps and planners. For autonomous navigation in a mapped
environment, the global costmap also receives a previously generated map of the
environment. ANS receives data from the odometry system as well. The odometry data is
used to determine the current position of the wheelchair and track progress towards the
goal during navigation. Autonomous navigation in unmapped environments operated in
similar manner to autonomous mapped navigation with one key exception; the global
planner is not loaded with a prebuilt map and must rely exclusively on current laser scan
data for path planning.
11
Figure 1. Block diagram of ANS.
Sensor System
The sensor system plays a critical role in ANS’s ability to navigate and localize
properly. The sensor system collects data about the environment such as the positions of
obstacles. The data collected by the sensor system is passed on to the ANS for use during
navigation. The sensors system consists of a LIDAR unit and proximity sensor.
12
LIDAR
The LIDAR unit is collect data about the position of the obstacles in the
surrounding environment. The LIDAR has a laser and receiver to collect this data. The
laser emits a pulsed beam of infrared light and the receiver absorbed the reflected beam.
The LIDAR determines the position of an obstacle (relative to the LIDAR) based on the
time taken by the laser burst to reflect and initial angle at which the light was emitted.
The data produced by the LIDAR unit is used by the ANS to generate a map of the
environment as well as obstacle detection during navigation.
Two different LIDAR units were used for the autonomous wheelchair. Early in
the project, the RPLIDAR A1 was used. The A1 is a 360-degree rotating laser and light
sensor produced by the Robopeak development team at Slamtec [23]. Slamtec later
released the RPLIDAR A2, which replaced the A1 for the autonomous wheelchair. The
A2 offered several improvements of the A1 (see Table 1). The A2 has a measurement
range of 8 meters while the A1 has a range of 6 meter [24]. The A2 doubles the sample
rate of the A1 from 2000 samples per second to 4000 samples per second. The A2 nearly
doubles the scan rate as well from 5.5 Hz to 10 Hz.
Table 1. Key Specifications for RPLIDAR A1 and A2 from Slamtec
Metric RPLIDAR A1 RPLIDAR A2
Range (meters) 6 8
Angular Range (degrees) 360 360
Sample Frequency (Hz) 2000 4000
Scan Frequency (Hz) 5.5 10
13
The LIDAR unit is placed on a wooden plank 1.32 meters above the ground and
attached to the wheelchair’s backrest (see Figure 2). The LIDAR’s mounting position
allows the sensor to ray-trace walls and tall furniture from a position slightly above a
seated individual’s head. The LIDAR is collects obstacle data in a 2D plane at its
mounting height, meaning it is unable to detect obstacle above or below that plane.
Proximity sensors maybe incorporated into the design to detect obstacles missed by the
LIDAR.
Proximity
A variety of near-range proximity sensors exist that can be used to detect
obstacles missed by the LIDAR. One such sensor is the HC-SR04 ultrasonic ranging
sensor. The HC-SR04 is low-cost sensor with a range of 2 cm to 200 cm. The HC-SR04
can be mounted on the autonomous wheelchair at several locations to detect obstacles.
A test case was conducted to demonstrate the proximity sensor’s ability to detect
an obstacle missed by the LIDAR. The wheelchair was obstructed by a chair and black
bookcase, each placed 1 meter apart in front of the wheelchair (see Figure 3). The
bookcase is tall enough to detected by the LIDAR and appears on the local costmap (see
Figure 4). However, the chair is not detected and does not appear in the local costmap. If
the wheelchair were to navigate in the current setup, it would collide with the chair. The
addition of the HC-SR04 ultrasonic ranging sensor allows both obstacles to be detected
and placed on the local costmap (see Figure 5). Including more proximity sensors will
enable the system to detect a greater number of obstacles that may be missed by the
LIDAR.
14
Figure 2. Autonomous wheelchair with a LIDAR, rotary encoders, sip-and-puff device,
and user-interface.
15
Figure 3. A test case scenario where the autonomous wheelchair is obstructed by a chair
and bookcase.
16
Figure 4. Local costmap generated using LIDAR data alone. The bookcase is detected as
an obstacle, but the chair is not.
Figure 5. Local costmap generated using LIDAR and proximity sensor data. Both the
bookcase and chair are detected as obstacles.
17
Odometry System
The odometry system sends odometry data about the movements of the
wheelchair to the ANS. The odometry data is used by ANS to estimate the wheelchair’s
position and determine navigation path. The odometry system consists of the Arduino
MEGA 2560 and two rotary encoders.
Arduino is an open-hardware Atmel AVR based microcontroller unit (MCU)
platform ideal for rapid prototyping. Arduino software is developed in the C/C++
language, taking advantage of Arduino-compatible software libraries. An essential library
for the autonomous wheelchair is rosserial [9], an Arduino port of ROS data-structures
and API methods for serial communication to ANS using USB.
Two optical rotary encoders are affixed to the outside of the drive wheels of the
autonomous wheelchair (see Figure 2). The encoders measure rotational movements of
each wheel, allowing software running on Arduino to determine the movement of the
wheelchair. The rotary encoders are incremental, providing 400 ticks per turn resulting in
an angular resolution of 0.9 degrees.
Software on the Arduino uses incremental data from encoders to compute the
distance traveled by the autonomous wheelchair. The distance computation uses several
constants; the number of ticks per revolution of the rotary encoders (T = 400 ticks), the
axle length between two wheels (L = 0.525 meters), and the radius of each wheel (r =
0.115 meters). Δ𝑇𝐿 and Δ𝑇𝑅 denote the number of ticks since the last computation the left
and right encoder, respectively. A positive value for Δ𝑇𝐿 or Δ𝑇𝑅 signifies forward
movement while a negative value signifies backward movement. Distance traveled by the
left and right wheels can be determined by:
18
Δ𝐷𝐿 = 2𝜋𝑟Δ𝑇𝐿𝑇
Δ𝐷𝑅 = 2𝜋𝑟Δ𝑇𝑅𝑇
The relative distance moved by the wheelchair at its center point is found by:
Δ𝐷𝐶 =Δ𝐷𝐿 + Δ𝐷𝑅
2
The relative rotation of the wheelchair is found by:
Δ𝜃 =Δ𝐷𝑅 + Δ𝐷𝐿
𝐿
Lastly, the relative position of the wheelchair is found by:
Δ𝑥 = Δ𝐷𝐶 cos Δ𝜃
Δ𝑦 = Δ𝐷𝐶 sin Δ𝜃
The odometry system continuously sends the x- and y-position of the wheelchair
to the ANS at a frequency of 20 Hz.
Wheelchair Control System
The wheelchair control system enables ANS to send velocity commands to an
electric wheelchair to controls its movements. The wheelchair control system consists of
an electric wheelchair, Arduino Uno, and two digital-to-analog converters (DACs). The
Hoveround MPV5 wheelchair is used with no manufacturer add-ons. The MPV5 has a
maximum speed of 4 mph (1.788 m/s), a turning radius of 22.7” (0.5766 m), and
dimensions of 24” width (0.6096 m) and 38” length (0.9652m) [13]. It uses an analog
joystick for operation, which is connected to the logic board of the wheelchair via an 8-
pin insulation-displacement connector (IDC). The wheelchair’s logic board operates at 5
19
volts DC. As such, the voltages from the analog joystick for each axis range from 0V-5V,
with 2.5V representing the center.
The Arduino UNO and DACs are used to mimic the operation of the analog
joystick. The MCU connects to the ANS via rosserial and waits for velocity commands
generated by the local planner. When a velocity command is received, the MCU converts
these commands into an appropriate Left/Right and Up/Down values from 0 to 4,095.
Two 12-bit MCP4725-based DACs takes these digital values and convert them to
voltages ranging from 0V-5V. The DAC outputs are connected to the wheelchair’s logic
board via an 8-pin IDC, allowing the ANS to control the wheelchair.
User-Input Device
Sip-and-Puff devices are an assistive technology which rely on sensing air
pressure from the user’s mouth to generate signals that can be interpreted by hardware
and software applications. These devices can be mounted on a wheelchair with an
adjustable gooseneck or on the head using a headset.
For the prototype, the Sip/Puff Breeze with Gooseneck from Origin Instruments
was used [29] (see Figure 6). This is a USB Sip/Puff device which can operate as a
joystick, mouse, or keyboard. It is a class-compliant device thus able to operate on the
Linux operating system used with the ROS. The Sip/Puff Breeze was configured to act as
a keyboard. In this mode, a sip is interpreted as the SPACE key being depressed and a
puff as the ENTER key being depressed.
20
Figure 6. Breeze Sip-and-Puff device from [29].
User Interface
The interface was designed such that all commands are operable by only the
SPACE and ENTER keys. Upon loading the app, the user is presented with a vertical
menu. The menu items are predefined locations e.g. “Kitchen,” “Bedroom,” “Bathroom,”
“Front Door” (see Figure 7). The user sips to switch between the items and puffs to select
the item. Once a selection is made, the user is presented with a UI indicating that the
navigation is currently in progress and gives the user a “Cancel” button which they can
select by puffing (see Figure 8). If the navigation is canceled, the wheelchair aborts the
current navigation plan and the user is presented with the original menu of destinations.
21
Figure 7. Main menu of user-interface.
22
Figure 8. Navigation screen of user-interface
23
CHAPTER 5
EXPERIMENTAL DESIGN, RESULTS, AND DISCUSSION
Several experiments were designed to test the performance of the autonomous
wheelchair with different environmental conditions. First, the autonomous navigation
capabilities of the wheelchair were tested in a mapped environment. The BCI-based
navigation proposed by Iturrate et al. [8] was selected as a benchmark to show the
improvements of autonomous navigation. As such, the test environment was similar to
[8]’s Task 2 environment. In the next set of experiments, the usability of a user-interface
(UI) was examined for selecting destination for navigation in a mapped environment. In
the final set of experiments, the autonomous wheelchair was tested in an unmapped
indoor environment.
Mapped Navigation
To navigate in a mapped environment, a map of the environment was first
generated. This was accomplished by running the gmapping package of ROS while
manually moving the wheelchair in the environment to be mapped. The generated maps
were saved and used for mapped navigation. The mapped navigation experiments were
conducted for two test cases: Case 1 and Case 2.
Case 1
Case 1 was used to test the autonomous navigation capabilities of the wheelchair
in a mapped environment with only static obstacles. The static obstacles in this case were
24
walls and cardboard used to create several environment configurations. The environment
for Case 1 closely resembles [8]’s Task 2 (see Figure 9). As a result, the generated map
(see Figure 10) of environment is similar to [8]’s Task 2.
Figure 9. Map of environment used in [8]’s Task 2.
Figure 10. Generated map of environment used for Case 1 showing distances and
approximate path of travel in trials.
Evaluation metrics. For an apt comparison, the evaluation metrics used to
measure the performance of the autonomous wheelchair were equivalent to the evaluation
metrics used by [8] for Task 2. The evaluation metrics are as follows:
25
• Task Success – degree to which the wheelchair reached the intended target, with
success defined as the chair stopping within 1 square meter of the target. ‘1’
represents successful navigation and ‘0’ represents failure.
• Path length – distance traveled by the wheelchair (in meters).
• Time – time taken by wheelchair to reach destination from origin (in seconds).
• Path length optimality ratio – ratio of the optimal path length and the actual path
length for a given trial. An optimal path is the shortest path to the destination
without collisions. The optimal path length was determined by manual
measurement of a path, which was 42.7 meters.
• Time optimality ratio – ratio of the optimal time and the actual time. An optimal
time is determined by dividing the length of the optimal path (42.7 m) by the
average speed of 0.375 m/s.
• Collisions – number of collisions between the wheelchair and any obstacles
during the trial.
Results. For Case 1, ten trials were conducted to test the navigation system of the
autonomous wheelchair. During each trial, the wheelchair was moved to the starting
position and given the goal of position down a hallway and to the left around a corner
(see Figure 10). A summary of the results from these trials is shown in Table 2. For
comparison, the results from [8]’s Task 2 are presented in Table 3.
Table 2. Results for Case 1
Min Max Mean Std.
Task Success 1 1 1 0
Path Length (m) 43.2 44.4 43.8 0.323
Time (s) 110 130 117 6.28
Path Opt. Ratio 1.01 1.04 1.03 0.00756
Time Opt. Ratio 0.966 1.14 1.03 0.0552
Collisions 0 0 0 0
26
Table 3. Results for [8]’s Task 2
Min Max Mean Std.
Task Success 1 1 1 0
Path Length (m) 37.5 41.4 39.3 1.3
Time (s) 507 918 659 130
Path Opt. Ratio 1.10 1.22 1.16 0.02
Time Opt. Ratio 2 4 2.75 0.28
Collisions 0 0 0 0
Discussion. The results for Case 1 show a large variance in the time metric with a
difference of 20 seconds between the best and worst time. The variation in time can be
attributed to path-planning of the ANS, specifically the local planner. The variation in
time results from navigating the corner. The local planner follows a greedy algorithm
which caused the wheelchair to follow the shortest path when making the turn around the
corner. During the turn, the footprint of the wheelchair crossed the buffer cloud around
the wall. When this occurs, the ANS drastically reduces the speed of the wheelchair to
avoid a potential collision. The variation in navigation time depends on how close the
wheelchair navigated to the corner as well as the velocity reduction by the ANS.
Navigation times can be normalized by reducing the size of the buffer cloud around
obstacles. However, this solution also reduces the safety of navigation since the
possibility of a collision increases. Consistency in navigation times can also be achieved
by reducing the reward for the local planner’s greedy algorithm. The drawback of this
solution is more inefficient navigation paths and reduction in the ANS’s ability to
account for unexpected changes in the environment. The results for other metrics are
consistent.
27
A comparison with the results for [8]’s Task 2 demonstrate the strength of
autonomous navigation. The average navigation time for the trials reported in [8] was 659
seconds, whereas the average time for the Case 1 trials was 117 seconds. The standard
deviation was lower as well, from 130 to 6.28 seconds. A direct comparison to [8] work
is not appropriate since one must account for BCI commands. [8]’s work required the
user to give navigation commands at various points of the task using BCI device.
Nevertheless, the advantages of autonomous navigation are clear.
Case 2
Case 2 was used to test the performance of the autonomous wheelchair in a more
complex in environment with static and dynamic obstacles. The two static obstacles were
set up in a hallway at opposite walls 1.88 meters apart (See Figure 11). The dynamic
obstacle was a person walking into the expected path of the wheelchair as it passed the
second static obstacle. The navigation path for Case 2 was more complex as well when
compared to Case 1 (see Figure 11). The wheelchair maneuvered in a more compact
space into order to successfully navigate to the destination.
Figure 11. Mapped environment with measurements and approximate path for Case 2.
Evaluation Metrics. The evaluation metrics for Case 2 are different from the ones
used in Case 1 due to the nature of Case 2. The presence of the dynamic obstacle
28
presented a challenge to measuring path length reliably, since it was difficult to determine
how the wheelchair would reroute its path to avoid the dynamic obstacle. The metrics
used to determine the responsiveness of the wheelchair to dynamic obstacles are as
follows:
• Task Success – degree to which the wheelchair reached the intended
target, with success defined as the chair stopping within 1 square meter of
the target. ‘1’ represents successful navigation and ‘0’ represents failure.
• Time – time taken by wheelchair to reach destination from origin (in
seconds).
• Speed – average speed calculated by taking the ratio of expected distance
and the elapsed time.
• Collisions – number of collisions between the wheelchair and any
obstacles during the trial.
Results. For Case 2, five trials were conducted to test the responsiveness of the
navigation system in an environment with dynamic and static obstacles (See Figure 11).
Results from these trials are shown in Table 4.
Table 4. Results for Case 2
Trial Task Success Time (s) Speed (m/s) Collisions
1 1 61 0.258 0
2 1 44 0.357 0
3 1 32.3 0.486 0
4 1 32.7 0.480 0
5 0 - - -
Discussion. The wheelchair successfully navigated around the dynamic obstacle
in 4 of the 5 trials for Case 2. During the final trial, the wheelchair failed to localize its
starting position on the map. This likely occurred due to a difference between the
29
environment when the map was first generated and when the wheelchair attempted to
localize. The issue can be results by moving the wheelchair around the staring position,
thus giving the AMCL package a better chance to localize properly.
As with Case 1, there was a notable variation in time, with 27.8 seconds between
the best and worst time. Again, the local planner’s greedy algorithm caused the
wheelchair to navigate too close to an obstacle when navigating around a corner. The
issue is exaggerated in Case 2 by the complexity of the path and well as the dynamic
obstacle. The distance at which the dynamic obstacle was perceived effected the
navigation time. If the dynamic obstacle was detected at a greater distance, the local
planner had a larger navigation window to reroute itself. A late detection of the dynamic
obstacle meant that the wheelchair may stop mid-navigation to allow the local planner to
create a new plan.
User-Interface
User-Interface was designed as an add-on to mapped navigation capabilities of the
autonomous wheelchair. The UI served as means for the user to select a destination
within a mapped environment. The map and environment from Case 2 were used to test
the UI.
Evaluation Metrics
Two metrics were developed specially to test the usability of the UI. These
metrics are as follows:
• Selection time – time when the user started to make a destination selection
to the wheelchair response (in seconds).
• Rerouting time – time to cancel navigation en route and select new
destination (in seconds).
30
Results
Ten trials where conducted to measure the effect, if any, of the sip-and-puff
device and user-interface on the ANS. The results from the trials are shown in Table 5.
Another ten trials were conducted to test the usability of the sip-and-puff device and the
UI. Results from these trials are shown in Table 6
Table 5. Results for ANS with Sip-and-Puff Device and User-Interface
Min Max Mean Std.
Task Success 1 1 1 0
Distance Traveled (m) 10.45 10.84 10.643 0.1216
Time (s) 24.91 33.80 28.556 3.2942
Speed (m/s) 0.3124 0.4239 0.3769 0.04103
Path Length (m) 10.26 10.26 10.26 0
Path Opt. Ratio 1.0185 1.0565 1.0373 0.01185
Time Opt. Ratio 1 1.3569 1.1464 0.1322
Collisions 0 0 0 0
Table 6. Results for User-Interface
Min Max Mean Std.
Selection Time (s) 2.31 16.54* 6.307 4.5014
Rerouting time (s) 2.69 7.57 4.865 1.9031
* Human error: Incorrect destination was selected.
Discussion
The map and path from Case 2 were used to determine any effect of the sip-and-
puff device and UI on the autonomous navigation system. The results indicate that the
sip-an-puff and UI have negligible impact on the autonomous navigation system. An
31
average selection time of the 6.307 seconds is added to the navigation, slightly increasing
the overall time for navigating to a destination. Two of the ten trials resulted in above-
average times due to human error in the selection of the destination. An incorrect
destination was selected so the navigation was canceled and the correct destination was
selected leading to outstanding selection times in Table 6.
The amount of time from the user selecting a new destination while navigating to
a previously selected destination to the wheelchair starting navigation to the new
destination (called rerouting time) is also shown in Table 6. The average rerouting time
during the trials was 4.865 seconds, demonstrating the ease of use for the sip-and-puff
device and UI. The sip-and-puff device combined with the user interface further
improves the accessibility of the autonomous wheelchair.
Unmapped Navigation
Unmapped indoor navigation requires the autonomous wheelchair to navigate to
destinations without the aid of a prebuilt map of the environment. Since the range of the
wheelchair is limited by the range of the LIDAR sensor, the wheelchair can only navigate
to destinations within that range when using unmapped navigation. Navigation to
destination beyond the range of the wheelchair is possible by navigating to a series of
intermediary destinations within the range of the wheelchair. A test case of a pseudo-
shopping mall is used as a test to autonomous navigation in an unmapped indoor
environment. Images of storefronts for Macy’s, Forever21, and JCPenny are placed at
various points in an indoor environment to serve as intermediary destinations.
The first destination (Macy’s) is placed 3 meters away from the starting point (see
Figure 12). At this range, the destination is clearly visible and within the range of the
32
RPLIDAR A2. The second destination (JCPenny) is placed 6 meters away from the
starting point (see Figure 13). At this range, the destination is partially visible to the
camera and beyond the range of the RPLIDAR A2. The third (and final) destination
(Forever 21) is placed 9 meters away from the starting point (see Figure 14).
33
Figure 12. First destination for shopping mall test case.
Figure 13. Second destination for shopping mall test case.
34
Figure 14. Third destination for shopping mall test case.
Evaluation Metrics
The following metrics were used to evaluate the efficacy of the navigation
system:
35
• Distance – Distance traveled by the wheelchair to reach destination (in
meters).
• Navigation Time – Time taken by wheelchair to navigate to destination (in
seconds).
• Speed – Distance traveled divided by navigation time (in meters per
second)
• Collisions – The number of collisions between the wheelchair and an
obstacle during navigation
Results
Ten trails were conducted to test the autonomous navigation system in an
unmapped indoor environment. During each trial, the final destination (Forever21) was
placed beyond the range of the wheelchair. The wheelchair navigated to the first
intermediary destination (Macy’s) (see Figure 16). At this point, the second intermediary
destination (JCPenny) was within range of the wheelchair (see Figure 17). The
destination (Forever 21) was in range for navigation (see Figure 18). Results for the ten
trials are shown in Table 7, 8, and 9.
36
Figure 15. Initial view of local costmap for unmapped navigation.
37
Figure 16. Local costmap showing first intermediary destination in range of wheelchair.
38
Figure 17. Local costmap showing second intermediary destination in range of
wheelchair.
39
Figure 18. Local costmap showing final destination in range of wheelchair.
Table 7. Results for Unmapped Navigation to First Intermediary Destination
Min Max Mean Std.
Distance (m) 3.27 4.1 3.706 0.22
Time (s) 9.75 12.59 11.25 0.84
Speed (m/s) 0.29 0.42 0.33 0.04
Collisions 0 0 0 -
40
Table 8. Results for Unmapped Navigation to Second Intermediary Destination
Min Max Mean Std.
Distance (m) 4.2 4.55 4.37 0.13
Time (s) 13.07 22.3 14.68 2.87
Speed (m/s) 0.19 0.34 0.31 0.04
Collisions 0 0 0 -
Table 9. Results for Unmapped Navigation to Final Destination
Min Max Mean Std.
Distance (m) 3.2 3.54 3.39 0.11
Time (s) 10.65 12.02 11.15 0.47
Speed (m/s) 0.27 0.33 0.30 0.02
Collisions 0 0 0 -
Discussion
The ANS successfully navigate to the first intermediary destination in 100% of
the trials. The system successfully navigated the second intermediary and final
destinations in 90% of the trails. During trial 4 for second intermediary destination, the
sensor system detected false obstacles in its path and tried rerouting the wheelchair. The
ANS determined that no viable path was available and aborted the navigation. During
trail 6 for the final destination, the wheelchair moved too close to an obstacle and aborted
navigation to avoid a collision.
The average navigation time for the trials was approximately 11 seconds and there
were no collisions during the trails. The results show the robustness of the navigation
system and its capability to overcome the challenges of unmapped autonomous
navigation.
41
CHAPTER 6
CONCLUSION
The autonomous wheelchair capable of navigating in both mapped and unmapped
indoor environments was proposed to address the challenges of reduced autonomy and
limited mobility faced by individuals with motor-neurodegenerative diseases and SPIs.
The wheelchair used low-cost hardware and open-source software, making it a
compelling solution.
Ten trials were conducted to test the autonomous navigation capabilities of the
wheelchair in indoor mapped environment. The wheelchair successfully navigated to the
destination in 100% of the trials with no collisions, while avoiding static obstacles in real
time. The navigation time was a significant improvement over a comparable solution.
Five more trials were conducted to test the performance of the wheelchair in a more
complex indoor mapped environment with both static and dynamic obstacles. The
wheelchair successfully navigated to the destination in 80% of trials with no collisions,
showing its ability to detect and avoid both static and dynamic obstacles in real time.
which shows promise of operating in unpredictable
The wheelchair was further improved by adding a Sip-and-Puff device and UI.
Over ten trials, the results show that the improvement had negligible effects on the ANS.
Ten trials were conducted to measure the usability of the Sip-and-Puff device and UI.
42
The results showed short selection and rerouting times, demonstrating the ease of use for
selecting destinations in mapped indoor environments.
Ten trials were conducted where the wheelchair navigated to destination beyond
its range in an unmapped indoor environment by navigating to intermediary destinations.
Autonomous navigation to the first intermediary was successful in 100% of the trials and
90% of trials for second intermediary and final destinations. The results show the
autonomous wheelchair can navigate successfully in both mapped and unmapped
environments.
The results are promising and offer ample opportunities to improves the lives of
many people by increasing their mobility and autonomy. Indoor mapped navigation and
UI are ideal for use in a private home or commonly used locations. The map of the
location would be generated on first-time and subsequently used of later navigation. The
autonomous wheelchair’s ability to detect and avoid both static and dynamic obstacles
ensures the safety of the user as well others in the environment. Unmapped navigation
may be used when navigating in unfamiliar environments. If future navigation the
environment is required, the map generated during unmapped navigation can be saved for
future use. More testing in realistic environments is required to further develop the
proposed system and deliver it to people who can most benefit from it.
43
REFERENCES
[1] C. P. DiGlovine, “Electric Wheelchair,” Britannica. [Online]. Accessed March 31,
2017. Available at: https://www.britannica.com/topic/electric-wheelchair
[2] “Amyotrophic Lateral Sclerosis (ALS) Fact Sheet,” NINDS, Publication date
June 2013. NIH Publication No. 13-916. [Online]. Accessed March 31, 2017.
Available at:
http://www.ninds.nih.gov/disorders/amyotrophiclateralsclerosis/detail_ALS.htm
[3] J. S. Nguyen, S. W. Su, and H. T. Nguyen, “Experimental Study on a Smart
Wheelchair System Using a Combination of Stereoscopic and Spherical Vision,”
35th Annual International Conference of the IEEE EMBS. Osaka, Japan, 3 – 7
July, 2013.
[4] C. Urdiales, A. Poncela, I. Sanchez-Tato, F. Galluppi, M. Olivetti, and F.
Sandoval, “Efficiency based reactive shared control for collaborative human/robot
navigation,” Proceedings of the 2007 IEEE/RSJ International Conference on
Intelligent Robots and Systems. San Diego, CA, USA, Oct 29 - Nov 2, 2007.
[5] United Nations, Department of Economic and Social Affairs, Population Division
(2013). World Population Ageing 2013, pp. xi-xiii, ST/ESA/SER.A/348
[6] N. D. Wee-Kiat, S. Ying-Wei, and G. Sing-Yau, “Development of an
Autonomous BCI Wheelchair,” IEEE. 2014.
[7] C. H. Kuo, H. W. Yeh, C. E. Wu, and K. M. Hsiao, “Development of
Autonomous Robotic Wheelchair Controller Using Embedded Systems,” The
33rd Annual Conference of the IEEE Industrial Electronics Society (IECON)
Nov. 5-8, 2007, Taipei, Taiwan.
[8] I. Iturrate, J. M. Antelis, A. Kubler, and J. Minguez, “A Noninvasive Brain-
Actuated Wheelchair Based on a P300 Neurophysiological Protocol and
Automated Navigation,” IEEE Transactions on Robotics, vol. 25, no. 3, pp. 614-
627, June 2009.
[9] “rosserial arduino,” ROS Wiki. [Online]. 24-Oct-2011. Available at:
http://wiki.ros.org/rosserial_arduino
[10] “About ROS,” ROS. [Online]. Available at: http://www.ros.org/about-ros/
[11] E. W. Dijkstra, Number. Math. (1959) 1: 269. doi:10.1007/BF01386390
44
[12] D. Fox, W. Burgard, S. Thrun, “The Dynamic Window Approach to Collision
Avoidance,” IEEE Robotics & Automation Magazine, vol. 4, no. 1, pp. 23-33,
March 1997.
[13] “MPV5,” Hoveround. [Online]. Available at:
https://www.hoveround.com/mobility-solutions/power-
wheelchairs/standard/mpv5
[14] “ROS Introduction,” ROS Wiki. [Online]. Available at:
https://wiki.ros.org/ROS/Introduction
[15] “ROS Concepts,” ROS Wiki. [Online]. Available at:
https://wiki.ros.org/ROS/Concepts
[16] “Packages,” ROS Wiki. [Online]. Available at: https://wiki.ros.org/Packages
[17] “gmapping,” ROS Wiki. [Online]. Available at: https://wiki.ros.org/gmapping
[18] “amcl,” ROS Wiki. [Online]. Available at: https://wiki.ros.org/amcl
[19] “move_base,” ROS Wiki. [Online]. Available at: https://wiki.ros.org/move_base
[20] “Nodes,” ROS Wiki. [Online]. Available at: https://wiki.ros.org/Nodes
[21] “Master,” ROS Wiki. [Online]. Available at: https://wiki.ros.org/Master
[22] “Topics,” ROS Wiki. [Online]. Available at: https://wiki.ros.org/Topics
[23] “RPLIDAR A1,” Slamtec. [Online]. Available at:
https://www.slamtec.com/en/Lidar/A1
[24] “RPLIDAR A2,” Slamtec. [Online]. Available at:
https://www.slamtec.com/en/Lidar/A2
[25] “How many people use assistive devices?” Eunice Kennedy Shriver National
Institute of Child Health and Human Development. [Online]. Accessed March 31,
2017. Available at: www.nichd.nih.gov
[26] L. Lu, J. T. Wen, “Human-Robot Cooperative Control for Mobility Impaired
Individuals,” American Control Conference. July 2015.
[27] T. F. Bastos-Filho, et. al., “Towards a New Modality-Independent Interface for a
Robotic Wheelchair,” IEEE Transactions on Neural Systems and Rehabilitation
Engineering, vol. 22, no. 3, pp. 567-584, May 2014.
[28] S. P. Levine, D. A. Bell, L. A. Jaros, R. C. Simpson, Y. Koren, J. Borenstein,
“The NavChair Assistive Wheelchair Navigation System,” IEEE Transactions on
Rehabilitation Engineering, vol. 7, no. 4, pp. 443-451. December 1999.
45
[29] “Sip and Puff Switch Solutions,” Origin Instruments. [Online]. Accessed March
31, 2017. Available at: www.orin.com/access/sip_puff/
[30] H. S. Grewal, A. P. Matthews, R. Tea, and K. George, “LIDAR-Based
Autonomous Wheelchair,” IEEE Sensors Applications Symposium (SAS), March
2017, Glassboro, NJ, USA.
[31] J. Biwas and M. Veloso, “WiFi Localization and Navigation for Autonomous
Indoor Mobile Robots, 2010 IEEE International Conference on Robotics and
Automation (ICRA), May 3-7, 2010, Anchorage, AK, USA.
[32] D. I. Vilaseca and J. I. Giribet, “Indoor Navigation using WiFi signals, 2013
Fourth Argentine Symposium and Conference on Embedded Systems
(SASE/CASE), Aug. 14-16, 2013, Buenos Aires, Argentina.
[33] A. H. Salamah, M. Tamazin, M. A. Sharkas, and M. Khedr, “An Enhanced WiFi
Indoor Localization System Based on Machine Learning, 2016 International
Conference on Indoor Positioning and Indoor Navigation (IPIN), October 4-7,
2016, University of Alcal, Madrid, Spain.
[34] L. C. de Oliveira et al., “Indoor Navigation with Mobile Augmented Reality and
Beacon Technology for Wheelchair Users, 2017 IEEE EMBS International
Conference on Biomedical & Health Informatics (BHI), Feb. 16-19, 2017,
Orlando, FL, USA.
