View
219
Download
3
Category
Preview:
Citation preview
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
1
Robotic Telepresence
Marcelo H Ang Jr, mpeangh@nus,edu.sg & Wong Hong Yang
Dept of Mechanical Engineering, National University of Singapore
Lim Ser Yong
Automation Division, Gintic Institute of Manufacturing Technology
5 Jan 2001
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
2
Background
• Robots as “super” tools for humans– vs complete autonomy
• Tasks unpleasant/unsuitable for humans– hazardous environments– tedious, low-level and high-level skills
• Tasks where robots can improve performance– better accuracy and repeatabilty
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
3
The Teleoperator
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
4
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
5
Challenges
• Telepresence– as if operator is at remote site– all five senses are available
• sight, touch, sound, smell, taste
– real-time feedback of sensation• via Internet?
• Robotic manipulator with human dexterity
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
6
• What are the limits in terms of tasks– that can be operated “telepresently”?
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
7
Technologies
Position tracking
3-D position tracking:Magnetic, Ultrasonic
or optical sensors
Visual feedback
Head-MountedDevices: LCD or CRT,
CAVE systems
Aural feedback 3-D sound systems
Touch feedback Pneumatic orvibrotactile systems
Force feedback Force reflectingexoskeletons, gloves
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
8
Haptic Devices*
*Phantom devices, Sensable Technologies
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
9
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
10
System Overview
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
11
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
12
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
13
Robot ArmHEM
HumanOperator
Master Environment
Slave Environment
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
14
Position Tracking using Polhemus Position Tracking using Polhemus FASTRAK Magnetic Position SensorFASTRAK Magnetic Position Sensor
FASTRAK receiver in
HMD
FASTRAK receiver on
velcro
FASTRAK transmitter
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
15
• 3 translation and 3 rotation data (Euler angles) are obtained (at receiver from transmitter).
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
16
Head-Eye Module (HEM, V 1)Head-Eye Module (HEM, V 1)• Separated into 2 parts: the Eye Module and the Head
Module.• Motion provided by servo motors, directly coupled to
drive each joint.• Rotation is provided in 2 orientation frame axes (x,y)
and 1 reference frame axis (Z)• Motion control provided by a 3 axes motion control
ISA bus PC card with user programming library
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
17
Eye Module
Leadscrew
Camera holder
Linear guides
“Carriage”
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
18
Head ModuleCoincidenceof rotating
axes
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
19
Robot (Slave) Arm and GripperRobot (Slave) Arm and Gripper• Movement is done in tool mode.
(Cartesian)• Orientation is defined by a series of 6
numbers (orientation and approach vectors)
• A data record of 9 numbers (3 position + 6 orientation values) is passed to the robot controller through RS232 interface from PC.
+x
+y
+z
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
20
System In OperationSystem In Operation
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
21
Robotic Telepresence System, V2
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
22
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
23
Telepresence with Haptics
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
24
SummarySummary• Successful integration of hardware and
software components. Only a short period of time is needed for operators to feel immersed and gain skill in operator the robot arm.
• Possible applications : The Armed Forces, Underwater dredging and salvage operations, Telesurgery, Education
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
25
Future Work
• Improved Head Eye Module– vergence control– compact
• Inherent delay in Internet
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
26
But are the existing Internet Protocols enough for communications between machines?• Remote Monitoring and Control
• Machines as Web Controllable Objects
• http? ftp? telnet?
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
27
RequirementsRequirements• a system for different machines to be controlled anda system for different machines to be controlled and monitored on different platforms monitored on different platforms
• a system that links up the various software packagesa system that links up the various software packages available now available now
• an open standardan open standard
• an object (component) oriented system, OOSan object (component) oriented system, OOS
Machines Accessed Via InternetMachines Accessed Via Internet
architecturearchitecture
Off the shelf components working together (Plug and Play for software and hardware interoperability)
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
28
Machines Accessed Via InternetMachines Accessed Via Internet
architecturearchitecture
SolutionSolution• 2 new layers between objects in the system2 new layers between objects in the system Machine Command Abstraction (MCA) and Machine Command Abstraction (MCA) and Message Communication Protocol (MCP) Message Communication Protocol (MCP)
• MCA bridges the differences between the functional languageMCA bridges the differences between the functional language of different machines (device independent) of different machines (device independent)
• MCP defines a standard way of communicating betweenMCP defines a standard way of communicating between different objects using text messages (platform independent) different objects using text messages (platform independent)
• Open Object Oriented StandardOpen Object Oriented Standard
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
29
Machines Accessed Via InternetMachines Accessed Via Internet
architecturearchitecture
Parallel/AnalogyParallel/Analogy• A MAVI compliant system is just like an FTP systemA MAVI compliant system is just like an FTP system
• Any FTP client can connect to any FTP server and send Any FTP client can connect to any FTP server and send commands and receive feedbacks from the servercommands and receive feedbacks from the server
• Similarly, any device connected to a MAVI compliant server Similarly, any device connected to a MAVI compliant server can be controlled and monitored by any MAVI clientcan be controlled and monitored by any MAVI client
• User Interface will be independent of device and platformUser Interface will be independent of device and platform
• Can be expanded for data transfer (example between smart Can be expanded for data transfer (example between smart agents or feedback sensors)agents or feedback sensors)
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
30
Machines Accessed Via InternetMachines Accessed Via Internet
architecturearchitecture
MCAMCA• The Machine Command Abstraction translate different The Machine Command Abstraction translate different machine commands that are vendor specific to a standard set of machine commands that are vendor specific to a standard set of commandscommands
• Free programmers from having to rewrite their User Interface Free programmers from having to rewrite their User Interface for different devices (one UI many devices)for different devices (one UI many devices)
• Allows different UIs to be used on a single device (one device Allows different UIs to be used on a single device (one device many UIs)many UIs)
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
31
MCA - example commands for WCOs• On/Off
• axis label 5 m
• power label 90%
• start/stop
• get status
• etc.
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
32
Machines Accessed Via InternetMachines Accessed Via Internet
architecturearchitecture
MCPMCP
• The Message Communication Protocol defines a standard The Message Communication Protocol defines a standard language to be used in the TCP/IP pipe language to be used in the TCP/IP pipe
• Follows the widely accepted XML formatFollows the widely accepted XML format
• By formatting all communication messages in the TCP/IP pipe in By formatting all communication messages in the TCP/IP pipe in textual XML format, it makes it easy to understand and textual XML format, it makes it easy to understand and implement, platform and programming language independentimplement, platform and programming language independent
• Secondary communication pipes can Secondary communication pipes can be established in real time through be established in real time through MCP for binary communication and MCP for binary communication and encrypted messagesencrypted messages
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
33
MAVI Server
An ExampleAn Example
InterfaceInterfaceDriverDriver
UserUserInterfaceInterface
Vendor BVendor BDeviceDevice
Vendor AVendor ADeviceDevice
MachineMachineDriverDriver
MachineMachineDriverDriver
MAVI Client
MAVI ServerMCP
MCP
MCA
MCA
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
34
Machines Accessed Via InternetMachines Accessed Via Internet
architecturearchitecture
FeaturesFeatures• Designed for the Internet and Intranet.Designed for the Internet and Intranet.
• Based on TCP/IPBased on TCP/IP
• Simple but expandable and scalableSimple but expandable and scalable
• Independent ofIndependent of
• Machine VendorsMachine Vendors
• Operating SystemOperating System
• Programming LanguageProgramming Language
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
35
With MAVI…..
• Plug and Play WCO s
• Pervasive
• Home/Factory Use
• No programming experience
• Vendor/Hardware/Software independent
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
36
Machines Accessed Via InternetMachines Accessed Via Internet
architecturearchitecture
ScenariosScenarios• User designs a prototype at home. Uses a shareware MAVI User designs a prototype at home. Uses a shareware MAVI client to log into an Internet Rapid Prototyping shop. Remotely client to log into an Internet Rapid Prototyping shop. Remotely control, monitor and produces his prototype. Product delivered control, monitor and produces his prototype. Product delivered to him.to him.
• Factory supervisor on leave. Uses the Internet and a MAVI Factory supervisor on leave. Uses the Internet and a MAVI client to log into his factory machines while overseas. Check on client to log into his factory machines while overseas. Check on the machine up and down time and number of products the machine up and down time and number of products produced.produced.
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
37
Case Study 1 - TelerobotCase Study 1 - Telerobot
http://telemfg.eng.nus.edu.sgFeatures:Features:
• Control and monitor robot from anywhere in the worldControl and monitor robot from anywhere in the world
• Virtual Model for faster feedbackVirtual Model for faster feedback
• Easily adapted to any other robotEasily adapted to any other robot
• User Interface can be changed easilyUser Interface can be changed easily example force feedback joystick example force feedback joystick
• Additional components can be upgraded or addedAdditional components can be upgraded or added examples Security, Encryption, Image Processing, examples Security, Encryption, Image Processing, Object Avoidance etc Object Avoidance etc
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
38
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
39
Case Study 2 – i2hCase Study 2 – i2hhttp://mavi.mpe.nus.edu.sg/http://mavi.mpe.nus.edu.sg/
Internet Integrated HomeInternet Integrated Home
• Joint project with NCB to connect the Singapore One HomeJoint project with NCB to connect the Singapore One Home to the Internet (completed in 2000) to the Internet (completed in 2000)
• Objective – To implement the control and monitoring of homeObjective – To implement the control and monitoring of home devices via the Internet. The implementation must be simple devices via the Internet. The implementation must be simple and secured so to encourage market adoption. and secured so to encourage market adoption.
• Phase 1 – Implementation of some Web Controllable Objects,Phase 1 – Implementation of some Web Controllable Objects, (WCO) like Pan/Tilt WebCam and Table Lamps. (WCO) like Pan/Tilt WebCam and Table Lamps.
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
40
National University of SingaporeDept of Mechanical EngineeringSingapore 119260
Gintic Institute of Manufacturing Technology
41
Other work…..• Jini
– architecture based on Java for federating services in a distributed system
– requires Java virtual machine, language dependent
• CORBA– Common Object Request Broker defines an Interface Definition
Language (IDL) to create interfaces to objects– Requires language dependent ORB to work
• Microsoft’s (UPnP) Universal Plug N Play (newest)– Windows software in any device….
Recommended