D-Rex

Final Report

An Ho April 21, 14

University of Florida Department of Electrical and Computer Engineering

EEL 4665 – IMDL Instructors: A. Antonio Arroyo, Eric M. Schwartz

TA: Andrew Gray

 

2 | P a g e

Table  of  Contents  Abstract ........................................................................................................................................... 3  

Integrated System ............................................................................................................................ 3  

Mobile Platform .............................................................................................................................. 4  

Sensors ............................................................................................................................................ 6  

Special Speech Recognition System ............................................................................................... 6  

Vision .............................................................................................................................................. 8  

Behaviors ........................................................................................................................................ 9  

Conclusion ...................................................................................................................................... 9  

 

3 | P a g e

Abstract  A typical robot is quite cold and mechanical that usually just minds its own business, doing its assigned tasks. A pet like a dog or a cat, however, is much more active and engaging. A pet responses to its owner action with enthusiasm. The purpose of D-Rex is to try and imitate that by giving it a pair of eyes and voice that follow the owner commands.

Integrated  System  The robot use a Raspberry Pi for the high level decision and an expansion board called Gertduino (equivalent to an Arduino Uno) for the low level decision. Typical IR sensors, bump switches, motor controller, and servos are connected to the Gertduino. A camera and a Bluetooth headset are going to be connected to the Raspberry Pi. The block diagram below shows the preliminary components connection of the system.

Figure 1: Integrated system block diagram

Raspberry  Pi

Gertduino

Headset

Motor  Controller

Camera

Servos

4 | P a g e

Mobile  Platform  

Figure 2: Preliminary platform

The robot is a two wheel drive system with a compact base size that will divide into two levels. The reason is that the second level allows the camera to have a wider range of vision. The platform will be made primarily out of woods to keep cost down and since it is not doing mechanically intensive task, stronger material is unnecessary.

In the final design, the power circuit and the lipo battery are mounted on the bottom platform. The power circuit is made with perfboard and mounted with plastic spacer to leave some room underneath for wiring.

Figure 3: Bottom Platform (side view)

5 | P a g e

Figure 4: Bottom platform (power board view)

On the top platform, the Raspberry Pi is mounted with plastic spacers along side a USB powered mini speaker. At the front of the top platform is the camera mounted on two servo, allowing it to have 2-axes of movement.

Figure 5: Top platform

6 | P a g e

Sensors  The robot will have the IR sensors for proximity detection, mechanical switches for bump sensor, a camera as vision, and a headset for voice control.

The IR sensor is the typical Sharp sensor that provides an analog voltage level based on the distance of the object in front of it. It is typically very finicky so multiple of them will be used to get a more accurate detection.

For the bump sensor, a switch similar to the one in the Bird Buggy by Andrew Gray IMDL robot will be use in the back for when backing up.

The main sensor is the camera that will be attached to the Raspberry Pi. It will be used along with OpenCV library to detect different object and act upon it. The headset is an extra sensor that will be used to give command to the robot and demonstrate that it understood the owner.

Special  Speech  Recognition  System  What’s special about D-Rex is its ability to listen to the owner and response to the command given with its speech recognition system. Running inside D-Rex is the Pocketsphinx library. Pocketsphinx is an open source library for speech recognition. Pocketsphinx works by making use of hidden Markov acoustic models (HMMs) and an n-gram statistical language model. Pocketsphinx is a version of Sphinx that can be used on an embedded system, like the Raspberry Pi inside D-Rex.

The way the system works is shown below in figure 3.

Figure 6: Speech recognition processing steps

Record  

Write  to  .wav  

Read  .wav  Processing  

What  word  is  it?  

Send  command  to  micro-­‐controller  

7 | P a g e

The first step is to record a sound file with the microphone for a certain amount of time. As of this time, it is set to record for 2 seconds. Then, the program would output the data into a .wav file that is then read by the Pocketsphinx decode message function. The decoding would output a array of words that was spoken. By looking at which of the words was spoken, the program would then send a preselected command to the microcontroller to run a variety of action.

The process of getting a good audio signal into the program was quite a challenge. The summary of the selection of the microphone suitable for D-REX is as follow:

Figure 7: Microphone selection process

The first one to be used was a wired microphone, connected to a USB soundcard. This work extremely well, but D-REX needed a wireless microphone to be practical. Thus, the next attempt was with a Bluetooth headset. This proved challenging. The Bluetooth headset connected to the receiver with the Raspberry Pi just fine. Getting the microphone audio was not successful however. Next attempt is with a Pyle Pro wireless microphone system, a set similar to the one used by instructor at university. It works but is very noisy and give false recognition with the poor audio quality. The last attempt is with a Logitech H600 wireless microphone. This device work perfectly, it was a plug-and-play procedure.

In its current build, D-REX can understand the following instructions:

• GO STRAIGHT • GO BACK • GO LEFT • GO RIGHT • GO RANDOM • FIND GREEN/BLUE/RED (color) • FOLLOW GREEN/BLUE/RED (color)

8 | P a g e

It is a small list of instruction and definitely not indicative of D-REX full capabilities. Due to the nature of the university wireless network being inaccessible through the Raspberry Pi, D-REX can only accomplish those small tasks. If D-REX is able to connect to the Internet, it can potentially pull in various data, such as weather, time and date, even email content.

Vision  The speech recognition system is great to make D-Rex understand what the owner want but it would still be limited by mundane actions with just a set of motors and servos. To help aid D-Rex with more ability to do complex actions, a camera is also connected to the Raspberry Pi. By using the camera along with the powerful open source OpenCV library, D-Rex can accomplished much greater feat with its ability to see.

The vision program would first take in the data from the camera, which is in RGB color space format and convert it to the HSV format. Then it would do a blur filter on the image to reduce random noise. Afterward, depending on which color is being called, it would do a threshold on the image based on a preset value to capture only the color being seek.

At this point, based on what the camera is seeing, if the color being seek fill the screen by an amount of pixels greater than a threshold value, it would determine the x-y coordinate of the color and send that information to the microcontroller.

Figure 8: Finding a color with OpenCV

Figure 9: X-Y coordinate of the color

9 | P a g e

The Raspberry Pi is a small, embedded computer with an ARM processor so it does not have as much power as a normal laptop or desktop PC. This means the performance for OpenCV will generally be worse off. However, it is still capable of running at adequate frame rate that D-Rex can still see and track an object with a decent speed.

Behaviors  At start up, D-Rex will stand still waiting for instruction from the owner. The speaker will indicate when it is ready to accept a new command. A couple LED is being added to indicate this also, in case of the environment being to noisy to know when D-Rex is actually recording.

Based on the instruction received, a certain keyword will be sent to the microcontroller to indicate which program is being run. On the Raspberry Pi, a function will be called to complete the instruction. In this case, there are two possible functions to be called on the Raspberry Pi, a ‘Find Color’ and a ‘Follow Color’ function. Depend on which function, there are data being sent to the microcontroller based on the vision information, using UART.

Conclusion  D-Rex, with its special eyes and ears, can accomplish the task of acting like a real life pet. In its current form, D-Rex can understand a few commands. Using the camera with OpenCV library, it can do basic vision processing like looking for a specific color and track it. Given the Raspberry Pi limited power, a more complicated vision program might be too taxing to run at a good speed. However, there are many other ways to improve D-Rex capability.

In an environment where D-Rex can be connected to the Internet, its functionality can be vastly improved. Recently, there is a new program called Jasper that utilizes Raspberry Pi and Pocketsphinx to accomplish impressive task with voice command. Google Now and Siri from Google and Apple had also shown great success with what voice command can do. While Google Now, Siri, or Jasper understood voice command, they are immovable, unlike D-Rex. D-Rex is a full fledge robot that can accomplished even greater tasks than what available right now.