Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
1
Networking CoroWare Robot
Emily Blatz
Carly Hessler
Nowele Rechka
Draft 4
December 1, 2014
CNT 4104 Software Project in Computer Networks
Instructor: Dr. Janusz Zalewski
Software Engineering Program
Florida Gulf Coast University
Ft. Myers, FL 33965
2
1. Introduction
The CoroWare Robot (Fig. 1 and 2), which is further referred to as the CoroBot, is an
educational robotic device. Previously, work (Fig. 3 and 4) was done to control the CoroBot via
website and voice commands via the Kinect. The current project on this device is a continuation
of four known previous projects [1]-[4].The primary technologies we aim to use to this purpose
are web-based technologies as well as the drivers behind the XBox 360 controller (XInput [3]).
The purpose of extending the previous work is to provide more mobile capabilities to control the
CoroBot with. By mapping its controls to the XBox controller, using a computer (monitor,
mouse, and keyboard) or website will not be as necessary, reducing most of the hardware
components to options.
The benefits of this project include learning ways to extend and maintain a project with legacy
behind it, which is something that the members of this project have not experienced much before
this. This is a core part of being a Software Engineer so it is imperative that the team learn how
to do this.
Figure 1 - CoroWare CoroBot as given by the manufacturers
3
Figure 2 - CoroWare Robot at start of project (October 2014)
4
Figure 3 - GUI from previous project [2]
5
Figure 4 - CoroWare Robot with Kinect
6
2. Software Requirements Specification
The goal of this project is to repair and improve the relevant portions of the previous projects
[1]-[4] and then expand it with additional control capabilities. The original website is outdated
and the Kinect functions do not work consistently or correctly. Our aim for the current project is
to get the original website up and running again, debug and upgrade the Kinect functions, and
introduce controlling it via the XBox 360 controller.
2.1. Physical Diagram
As shown in Figure 5, the CoroBot is currently connected to a Kinect, monitor, keyboard,
and mouse that a user can use to control the CoroBot. Speakers can be attached to hear
CoroBot response sounds. These devices can be connected physically to the CoroBot or
wirelessly via the website. The CoroBot is already supplied with Power and Reset
buttons, battery power type switch, and a battery supply outlet in its standard design. For
this project, an XBox 360 controller is being added to enhance the capabilities to control
the CoroBot. The point of this extension is to provide a form of control that is mutually
exclusive to having the CoroBot connected to a monitor, mouse, and keyboard or through
the website. The user will be able to choose to incorporate almost any combination of
controls to operate the CoroBot, an example of this would be:
● voice commands via Kinect to engage the CoroBot, tilt the camera, and move the
CoroBot in a certain direction
● motion commands via the XBox controller to move the CoroBot in a certain
direction
● monitor the path the CoroBot is taking via the Kinect camera (that can be
connected directly to the CoroBot or streaming from the website).
7
Figure 5 - Physical Diagram of Networking CoroWare Robot
8
2.2 Context Diagram
As shown in Figure 6, which follows the physical diagram in Figure 5, the CoroBot
Software can receive commands from the XBox controller, mouse, keyboard, and Kinect
(via voice command) to change the CoroBot’s directional motion and Kinect camera tilt.
As shown on a Use Case Diagram in Figure 7, a user can:
● move the joysticks and press the buttons on the XBox controller
● use the mouse and keyboard to select and type options within the GUI
● issue voice commands via Kinect.
The CoroBot receives video and depth from the Kinect camera which can be viewed on a
monitor that is either physically connected to the CoroBot or streaming from the website.
The CoroBot outputs response sounds via speakers when a command is received.
9
Figure 6 - Context Diagram of Networking CoroWare Robot
10
Figure 7 - Use Case Diagram for CoroBot inputs
This use case diagram shows the interactions between the user and the system,
which affect the CoroBot. The user has the choice of inputting commands in three
ways: opening the GUI and using the mouse and keyboard, entering an action
through the Xbox controller, or giving a voice command via Kinect. Any one of
these three actions will give instructions for the CoroBot to execute.
11
2.3. Software Requirements
Following the CoroBot Software interactions with external devices, as shown in Figure 6,
software requirements can be formulated as stated below. It is assumed that the CoroBot
Software will boot when the CoroBot is running on battery power and the power button is
pressed. It is assumed that the CoroBot has a built-in capabilities for audio.
2.3.1. Input Requirements
2.3.1.1. The CoroBot Software shall handle and store a Direction and Distance
received via the mouse, keyboard, XBox controller, or Kinect voice command for
its next motion step.
2.3.1.2. The CoroBot Software shall handle and store an Angle received via the
mouse, keyboard, XBox controller, or Kinect voice command for its next tilt
move.
2.3.1.3. The CoroBot Software shall handle video and depth received from the
Kinect camera to display to the user.
2.3.1.4. The CoroBot Software shall handle a component’s power setting received
via the mouse, keyboard, XBox controller, or Kinect voice command.
2.3.2. Output Requirements
2.3.2.1. The CoroBot Software shall acknowledge that a Kinect voice command
was received with a response sound.
2.3.2.2. The CoroBot Software shall display to the Monitor the command it will
execute next before executing that command and until the next command is
acknowledged and displayed. As shown in Figure 3, the format of the displayed
command is: “Commands: [current command]”.
2.3.2.3. The CoroBot Software shall move the CoroBot in a Direction for a
Distance stored as per Requirement 2.3.1.1.
2.3.2.4. The CoroBot Software shall tilt the Kinect camera at an Angle stored as
per Requirement 2.3.1.2.
2.3.2.5. The CoroBot Software shall stream video and depth from the Kinect
12
camera to the Monitor or website.
2.3.2.6. The CoroBot Software shall engage or disengage a component’s power as
per Requirement 2.3.1.4.
13
3. Design Description
3.1. Software Architecture
As shown in Figure 8, which is consistent with the Context Diagram in Figure 6, there are
four core modules for the CoroBot Software, three of which handle the possible inputs
for the CoroBot to receive commands. The GUI allows users to input commands via
mouse and keyboard that move the wheels or Kinect (in the previous projects [1-4] the
interface also moved the arm, but this no longer functions). The Xbox Controller Module
allows users to press buttons and move the joysticks to move the wheels or Kinect, but
not with specific numerical value inputs (unlike the Kinect and GUI). The Kinect Module
allows for voice commands to control the wheels, as well as modify where its own
camera is facing.
14
Figure 8 - Software Architecture of the CoroBot Software
15
3.2. Detailed Design
3.2.1. Static Perspective
The static perspective of the design is shown in Figure 9. The CoroBot Software
is dependent on the use of several different control systems, including the Xbox
controller, voice commands, and a GUI. The GUI is dependent on inputs from the
mouse and keyboard.
Figure 9 - Class Diagram of CoroBot Software
16
3.2.2. Dynamic Perspective
The dynamic perspective of the design is shown in Figure 10. The CoroBot
receives input from four sources: the mouse, the keyboard, the Xbox controller,
and voice commands. Each input prompts the CoroBot to perform an action.
Figure 10 - Sequence Diagram of CoroBot inputs
17
4. Implementation and Testing
The CoroBot has been tested and proven to turn on for a short amount of battery power, respond
to a limited number of voice commands via the Kinect, as well as successfully move its wheels.
The video streaming functions properly and responds to tilt commands. Unfortunately, the
robotic arm does not respond to any currently working programs. As such, we will not focus on
the robotic arm as maintenance appears to be too complicated for a single semester. Being that
all of what is desired to function in the CoroBot does so, work has begun on mapping CoroBot
controls to the XBox 360 controller and expanding the existing set of Kinect voice commands.
CoroBot’s functionality can be proven with the following tests:
4.1. Motion commands and component power setting commands provided to CoroBot are
displayed to the monitor before execution.
4.1.1. Per requirement 2.3.2.2.
4.2. Using the XBox controller directional pad (Dpad) buttons to make the CoroBot move
forward (up button), backward (down button), left (left button), or right (right button).
4.2.1. Per requirement 2.3.1.1.
4.2.2. Per requirement 2.3.2.3.
4.2.3. Per requirement 2.3.2.4.
4.3. Using a voice command and ensuring the CoroBot performs a response sound.
4.3.1. Per requirement 2.3.2.1.
4.3.2. Per requirement 2.3.1.1.
4.3.3. Per requirement 2.3.1.2.
4.3.4. Per requirement 2.3.1.4.
4.4. Pressing an XBox controller button (Dpad buttons, A, B, X, Y, or triggers) and
ensuring the CoroBot performs a response sound.
4.4.1. Per requirement 2.3.2.1.
4.4.2. Per requirement 2.3.1.1.
4.4.3. Per requirement 2.3.1.2.
4.4.4. Per requirement 2.3.1.4.
4.5. Pressing an XBox controller letter button (A, B, X, or Y) and ensuring the
corresponding CoroBot component is enabled/disabled.
18
4.5.1. Per requirement 2.3.2.6.
4.5.2. Per Requirement 2.3.1.4.
4.6. Using the XBox controller triggers to move the Kinect camera up (right trigger) or
down (left trigger).
4.5.1. Per requirement 2.3.1.2.
4.5.2. Per requirement 2.3.1.3.
19
5. Conclusion
With the completion of the project, the CoroBot is now controllable via mouse, keyboard, XBox
controller, and voice command. Each control system can work exclusively of the others or in
conjunction with the others. Currently, the XBox controller allows for components like the
motors, arm function, camera, and depth sensor to be engaged or disengaged via the letter
buttons A, B, X, and Y respectively; the DPad provides motion control at a standard distance per
direction; and the camera can be tilted up or down via the right and left triggers respectively. The
CoroBot now responds to all received commands with a response sound and displays all received
commands to the monitor. The CoroBot video can be viewed on an attached monitor via the GUI
or remotely via its Twitch account.
The biggest difficulty in implementing the XBox controller was developing a timer to check on
the controller button states continuously without interfering with the existing CoroBot software.
A manual, non-static timer had to be developed that could check various buttons for press,
pressure, or direction. Another issue we faced was finding the previous projects’ website. Due to
the fact that the website that was supposed to provide another way to control the robot did not
exist any longer, it was not possible to upgrade it. In order to be able complete the main portion
of the project, the XBox control system, and still provide remote streaming via website, we chose
to re-create the web-streaming service that existed in previous projects by implementing a
Twitch web-streaming account on the CoroBot. A constant problem in this project was the
battery life of CoroBot. With the provided hardware, CoroBot charges on tether and then runs on
battery for only 2-3 hours. This is a difficult way to develop and test code, as there is no battery
status or warning of how long it will last.
Further work that can be done on this project include repairing the arm functions, which may
only be a hardware issue rather than a software issue. Within our working XBox controller code
we have provided some skeleton code to eventually integrate arm functions with the XBox
controller capabilities. An expansion suggestion would be to allow for multiple XBox controllers
to power certain sections of the robot; for example, one controller would be for motion and
camera controls while another would be for arm controls. Each section would be dedicated to a
particular controller index, so only player one could control motors for example. Another route
20
would be to focus on a new website to control CoroBot remotely, rather than attempting to
upgrade an expired website. Future projects would also greatly benefit from changing the power
supply, as the primary issues faced during this project stemmed from the lack of a power tether.
21
6. References
[1] A. Fernandez, V. Fernandez and T. Nguyen, "Robotic Control with Kinect Vision
Comprehensive Report," Florida Gulf Coast University, 2013. URL:
http://itech.fgcu.edu/faculty/zalewski/projects/files/FernandezCorowareWithKinectDraft2
[2] V. Fernandez, “CoroWare CoroBot Wireless Connectivity (Draft #8),” Florida Gulf Coast
University, 2012. URL:
http://itech.fgcu.edu/faculty/zalewski/projects/files/Victor_Corobot_V8.pdf
[3] S. Caguana, A. Meyers, “Networking CoroWare Robot (Draft 4),” Florida Gulf Coast
University, 2013.
[4] S. Caguana, V. Giannone, “CoroWare Robot: Networking and Remote Upload,” Florida Gulf
Coast University, 2014.
[5] Microsoft, “XInput Game Controller APIs (Windows),” Microsoft, 2014. URL:
http://msdn.microsoft.com/en-us/library/windows/desktop/hh405053(v=vs.85).aspx