-
7/30/2019 NUST 1 Introduction
1/102
The City College of New York 1
Jizhong Xiao
Department of Electrical Engineering
CUNY City [email protected]
Syllabus and Introduction
Advanced Mobile Robotics
-
7/30/2019 NUST 1 Introduction
2/102
The City College of New York 2
Outline Syllabus
Course Description Primary Topics Textbook and references
Contact Information
Introduction
What is a Robot?
Why use Robots?
Robot History Robot Applications
Robotics Research at CCNY
-
7/30/2019 NUST 1 Introduction
3/102
The City College of New York 3
Syllabus
Course Description This course is an in depth study of
state-of-the-art technologies and methods of mobile robotics.
The course consists of two components: lectures
on theory, and course projects. Lectures will draw from
textbooks and current
research literature with several readingdiscussion classes.
In project component of this class, students arerequired to
conduct simulation study toimplement, evaluate SLAM algorithms.
-
7/30/2019 NUST 1 Introduction
4/102
The City College of New York 4
Primary Topics Introduction Robotics history, applications
Locomotion/Motion Planning/Mapping How to reduce odometry
errors
Probabilistic Robotics Mathematic Background, Bayes Filters
Kalman Filters (KF, EKF, UKF) Particle Filters
SLAM (simultaneous localization and mapping)
Syllabus
-
7/30/2019 NUST 1 Introduction
5/102
The City College of New York 5
SyllabusTextbooks:
Probabilistic ROBOTICS, Sebastian Thrun, Wolfram Burgard,
DieterFox, The MIT Press, 2005, ISBN 0-262-20162-3.E-copy is
available
Reference Material: Introduction to AI Robotics, Robin R.
Murphy, The MIT Press, 2000,
ISBN 0-262-13383-0. Introduction to Autonomous Mobile Robots,
Roland Siegwart, Illah
R. Nourbakhsh, The MIT Press, 2004, ISBN 0-262-19502-X
Computational Principles of Mobile Robotics, Gregory Dudek,
Michael Jenkin, Cambridge University Press, 2000, ISBN
0-521-56876-5
Papers from research literature
-
7/30/2019 NUST 1 Introduction
6/102
The City College of New York 6
Syllabus
Contact Information Cell Phone: 138-14543174
E-mail:[email protected] Website:
http://robotics.ccny.cuny.edu
Expected outcomes: Knowledge
Abilities Be able to read technical papers
Be able to write technical papers
Be able to conduct independent research
mailto:[email protected]://134.74.16.73/http://134.74.16.73/mailto:[email protected]
-
7/30/2019 NUST 1 Introduction
7/102The City College of New York 7
Outline Syllabus
Course Description Primary Topics Textbook and references
Contact Information
Introduction
What is a Robot?
Why use Robots?
Robotics History Robot Applications
Robotics Research at CCNY
-
7/30/2019 NUST 1 Introduction
8/102The City College of New York 8
What is a robot?
Hollywoodsimagination
R2-D2
Star Wars
3PO
-
7/30/2019 NUST 1 Introduction
9/102
The City College of New York 9
What is a robot?
Definition: Websters Dictionary
An automatic device that performs functions
ordinarily ascribed to human beings
washingmachine = robot?
Robotics Institute of American A robot (industrial robot) is a
reprogrammable,
multifunctional manipulatordesigned to movematerials, parts,
tools, or specialized devices,through variable programmed motions
for theperformance of a variety of tasks.
-
7/30/2019 NUST 1 Introduction
10/102
The City College of New York 10
What is a robot?
By general agreement, a robot is:A programmable machine that
imitates the actions orappearance of an intelligent creatureusually
a human.
To qualify as a robot, a machine must be able to:1) Sensing and
perception: get information from its surroundings
2) Carry out different tasks: Locomotion or manipulation,
dosomething physicalsuch as move or manipulate objects
3) Re-programmable: can do different things4) Function
autonomously and/or interact with human beings
-
7/30/2019 NUST 1 Introduction
11/102
The City College of New York 11
Types of Robots
Robot Manipulators
Mobile Manipulators
http://www.egr.msu.edu/~tanjindo/mm..JPG
-
7/30/2019 NUST 1 Introduction
12/102
The City College of New York 12
Types of Robots
Humanoid
Legged robots
Underwater robots
Wheeled mobile robotsAerial Robots
Locomotion
-
7/30/2019 NUST 1 Introduction
13/102
The City College of New York 13
Mobile Robot Examples
Hilare II
http://www.laas.fr/~matthieu/robots/
Sojourner Rover
NASA and JPL, Mars exploration
http://www.laas.fr/~matthieu/robots/http://www.laas.fr/~matthieu/robots/
-
7/30/2019 NUST 1 Introduction
14/102
The City College of New York 14
Autonomous Robot Examples
http://www.spawar.navy.mil/robots/images/gsr.jpg
-
7/30/2019 NUST 1 Introduction
15/102
The City College of New York 15
Autonomous helicopter
Goal: To develop a vision-guided robot helicopterwhich can
autonomously carry out functions applicableto search and rescue,
surveillance, law enforcement,inspection, mapping, and aerial
cinematography, in anyweather conditions and using only on-board
intelligence
and computing power
http://www-2.cs.cmu.edu/afs/cs/project/chopper/www/haughton-do.html
http://www-2.cs.cmu.edu/afs/cs/project/chopper/www/haughton-do.htmlhttp://www-2.cs.cmu.edu/afs/cs/project/chopper/www/haughton-do.htmlhttp://www-2.cs.cmu.edu/afs/cs/project/chopper/www/haughton-do.htmlhttp://www-2.cs.cmu.edu/afs/cs/project/chopper/www/haughton-do.htmlhttp://www-2.cs.cmu.edu/afs/cs/project/chopper/www/haughton-do.htmlhttp://www-2.cs.cmu.edu/afs/cs/project/chopper/www/haughton-do.html
-
7/30/2019 NUST 1 Introduction
16/102
The City College of New York 16
Why Use Robots?
Application in 4D environments Dangerous
Dirty
Dull
Difficult
4A tasks Automation
Augmentation Assistance
Autonomous
-
7/30/2019 NUST 1 Introduction
17/102
The City College of New York 17
Why Use Robots? Increase product quality
Superior Accuracies (thousands of an inch, wafer-handling:
microinch) Repeatable precision Consistency of products
Increase efficiency Work continuously without fatigue Need no
vacation
Increase safety Operate in dangerous environment Need no
environmental comfort air conditioning, noise protection, etc
Reduce Cost Reduce scrap rate Lower in-process inventory
Lower labor cost Reduce manufacturing lead time
Rapid response to changes in design
Increase productivity Value of output per person per hour
increases
-
7/30/2019 NUST 1 Introduction
18/102
The City College of New York 18
Pre-History of Robots
Automata: a machine or control mechanismdesigned to follow
automatically apredetermined sequence of operations orrespond to
encoded instructions
-
7/30/2019 NUST 1 Introduction
19/102
The City College of New York 19
Automata
Europe
-
7/30/2019 NUST 1 Introduction
20/102
The City College of New York 20
Automata
Asia
-
7/30/2019 NUST 1 Introduction
21/102
The City College of New York 21
Robot History
1961 George C. Devol obtains the first U.S. robot patent,
No.
2,998,237.
Joe Engelberger formed Unimation and was the first tomarket
robots
First production version Unimate industrial robot is installed
in adie-casting machine
1962
Unimation, Inc. was formed, (Unimation stood for
"UniversalAutomation")
-
7/30/2019 NUST 1 Introduction
22/102
The City College of New York 22
Robot History
The patent and industrial robot
-
7/30/2019 NUST 1 Introduction
23/102
The City College of New York 23
Robot History
What an industrial robot must have?
1968 Unimation takes its first multi-robot order from
General Motors.
1966-1972 "Shakey," the first intelligent mobile robot
system
was built at Stanford Research Institute, California.
-
7/30/2019 NUST 1 Introduction
24/102
The City College of New York 24
Robot History
Shakey (Stanford ResearchInstitute)
the first mobile robot to beoperated using AItechniques
Simple tasks to solve: To recognize an object
using vision
Find its way to the object
Perform some action on the
object (for example, to pushit over)
http://www.frc.ri.cmu.edu/~hpm/book98/fig.ch2/p027.html
http://www.frc.ri.cmu.edu/~hpm/book98/fig.ch2/Shakey.150.jpg
-
7/30/2019 NUST 1 Introduction
25/102
The City College of New York 25
Shakey
-
7/30/2019 NUST 1 Introduction
26/102
The City College of New York 26
Robot History
1969 Robot vision, for mobile robot guidance, isdemonstrated at
the Stanford ResearchInstitute.
Unimate robots assemble Chevrolet Vegaautomobile bodies for
General Motors.
1970
General Motors becomes the first company touse machine vision in
an industrial applicationThe Consight system is installed at a
foundryin St. Catherines, Ontario, Canada.
-
7/30/2019 NUST 1 Introduction
27/102
The City College of New York 27
The Stanford Cart
1973-1979 Stanford Cart
Equipped with stereovision.
Take pictures fromseveral differentangles
The computer gaugedthe distance betweenthe cart and obstaclesin
its path
Hans Moravec
http://www.frc.ri.cmu.edu/users/hpm/
-
7/30/2019 NUST 1 Introduction
28/102
The City College of New York 28
The Stanford Cart
-
7/30/2019 NUST 1 Introduction
29/102
The City College of New York 29
Robot History
1978 The first PUMA (Programmable Universal Machine for
Assembly) robot is developed by Unimation forGeneral Motors.
1981 IBM enters the robotics field with its 7535 and
7565Manufacturing Systems.
1983 Westinghouse Electric Corporation bought Unimation,
Inc., which became part of its factory automationenterprise.
Westinghouse later sold Unimation toStaubli of Switzerland.
-
7/30/2019 NUST 1 Introduction
30/102
The City College of New York 30
Industrial Robot --- PUMA
-
7/30/2019 NUST 1 Introduction
31/102
The City College of New York 31
Installed Industrial Robots
Japan take the lead, why? Shortage of labor, high labor cost
-
7/30/2019 NUST 1 Introduction
32/102
The City College of New York 32
How are they used?
Industrial robots 70% welding and painting
20% pick and place
10% others
Research focus on Manipulator control
End-effector design
Compliance device
Dexterity robot hand
Visual and force feedback
Flexible automation
-
7/30/2019 NUST 1 Introduction
33/102
The City College of New York 33
Robot Arm Dexterity
-
7/30/2019 NUST 1 Introduction
34/102
The City College of New York 34
Robotics: a much bigger industry
Robot Manipulators
Assembly, automation
Field robots
Military applications Space exploration
Service robots
Cleaning robots
Medical robots
Entertainment robots
-
7/30/2019 NUST 1 Introduction
35/102
The City College of New York 35
Field Robots
-
7/30/2019 NUST 1 Introduction
36/102
The City College of New York 36
Field Robots
-
7/30/2019 NUST 1 Introduction
37/102
The City College of New York 37
Service robots
-
7/30/2019 NUST 1 Introduction
38/102
The City College of New York 38
Service robots
http://www.irobot.com/
iRobot Scooba Robot
iRobot Verro 600 Pool Cleaning Robot
-
7/30/2019 NUST 1 Introduction
39/102
The City College of New York 39
Your servant?
-
7/30/2019 NUST 1 Introduction
40/102
The City College of New York 40
What is AI
Knowledge representation
Understanding natural language
Learning Planning and problem solving
Inference
Search Vision
-
7/30/2019 NUST 1 Introduction
41/102
The City College of New York 41
Learning and Evolution
Learning
Skills vs Task (Map acquisition)
Learning Methods
Learning by instruction
Learning by imitation
Learning by skill transfer
Evolution and adaptation
-
7/30/2019 NUST 1 Introduction
42/102
The City College of New York 42
The early stage of AI
-
7/30/2019 NUST 1 Introduction
43/102
The City College of New York 43
The start of AI
-
7/30/2019 NUST 1 Introduction
44/102
The City College of New York 44
Autonomous and Intelligence
-
7/30/2019 NUST 1 Introduction
45/102
The City College of New York 45
The Honda Humanoid (1997)
-
7/30/2019 NUST 1 Introduction
46/102
The City College of New York 46
Humanoid
-
7/30/2019 NUST 1 Introduction
47/102
The City College of New York 47
Robot Applications
Manufacture Industry
Assembling
Automation
Biotechnology Micro/Nano manipulation
Sample Handling
Automated Analysis
-
7/30/2019 NUST 1 Introduction
48/102
The City College of New York 48
Robot Applications
Military Applications
-
7/30/2019 NUST 1 Introduction
49/102
The City College of New York 49
Military Applications DARPA Programs:(Defense Advanced Research
Projects Agency)
Tactical Mobile Robotics
-
7/30/2019 NUST 1 Introduction
50/102
The City College of New York 50
DARPA Grand Challenge
2004, no team entry completed thedesignated routeBest result:
Red Team (CMU) - 7.4 miles$1 million prize unclaimed
2005, 5 teams finished 132 miles
Results: Stanley (Stanford) - 6h 54mRed Team (CMU) - 7h 5mRed
Team 2 (CMU) - 7h 14m
$2 million prize awarded to stanford
Field test of autonomous ground vehicles.
Desert terrain featuring natural and man-made obstacles.The
route not revealed until 2 hours before the event begins.
-
7/30/2019 NUST 1 Introduction
51/102
The City College of New York 51
DARPA Urban Challenge
Autonomous ground vehicles maneuvering in a mock city
environment:executing simulated military supply missions while
merging into moving traffic,navigating traffic circles, negotiating
busy intersections, and avoiding obstacles.
November 3, 2007
-
7/30/2019 NUST 1 Introduction
52/102
The City College of New York 52
Robot Applications
Fire Fighting, Search and Rescue
-
7/30/2019 NUST 1 Introduction
53/102
The City College of New York 53
Space Applications
Robonaut: a humanoid robot that canfunction as an astronaut
equivalent forspacewalks.
NASA/DARPA Robonaut:
When will the Robonauts take over space travel?
http://vesuvius.jsc.nasa.gov/er_er/html/robonaut/robonaut.html
-
7/30/2019 NUST 1 Introduction
54/102
The City College of New York 54
Robot Applications
Mars Rovers: Spirit and Opportunity --- twin robotgeologists;
landed on Mars: Jan 3, and Jan 24,2004, and still alive, today!
http://marsrovers.jpl.nasa.gov/overview/
MARS Exploration:
-
7/30/2019 NUST 1 Introduction
55/102
The City College of New York 55
Robot Applications
Robots for Assistive Technology
Robotic-Assisted Surgery
-
7/30/2019 NUST 1 Introduction
56/102
The City College of New York 56
Robot Applications
Entertainment Industry
-
7/30/2019 NUST 1 Introduction
57/102
The City College of New York 57
Robot Applications
Entertainment Robots
Sony-Qrio
-
7/30/2019 NUST 1 Introduction
58/102
The City College of New York 58
Personal Robot?
Just as the personal computer is used for automated
information management even in households, robots can beused to
execute domestic tasks.
Manipulation of bits of information (PC)
Manipulation of physical objects (PR)
http://www.personalrobots.com
Architect re of Robotic S stems
-
7/30/2019 NUST 1 Introduction
59/102
The City College of New York 59
Architecture of Robotic Systems Mechanical Structure
Kinematics model
Dynamics model Actuators: Electrical, Hydraulic, Pneumatic,
Artificial Muscle Computation and controllers Sensors
Communications User interface Power conversion unit
Environmental
sensors
Motion
plannerController
Mechanical
Structure
Configuration
sensor
-
7/30/2019 NUST 1 Introduction
60/102
The City College of New York 60
Summary
Robotics--interdisciplinary research Mechanical design
Computer science and engineering
Electrical engineering
Cognitive psychology, perception and neuroscience
Research open problems Manipulation, Locomotion
Control, Navigation
Human-Robot Interaction
Learning & Adaptation (AI)
-
7/30/2019 NUST 1 Introduction
61/102
The City College of New York 61
Outline Syllabus
Course Description Primary Topics Textbook and references
Contact Information
Introduction What is a Robot?
Why use Robots?
Robot History
Robot Applications
Robotics Research at CCNY
Robotics Research at CCNY
-
7/30/2019 NUST 1 Introduction
62/102
The City College of New York 62
Dr. Jizhong Xiao, Associate Professor
Dept. of Electrical Engineering, City College of New York
Robotics Research at CCNY
R b i R h CCNY
-
7/30/2019 NUST 1 Introduction
63/102
The City College of New York 63
Dr. Jizhong XiaoAssociate Professor, Director of
Robotics and Intelligent Systems LabCenter forPerceptual
Robotics, Intelligent Sensors & Machines
(PRISM Center @ CCNY)Department of Electrical Engineering
The City College, City University of New York
Email:[email protected]:
http://robotics.ccny.cuny.edu
Robotics Research at CCNY
O
mailto:[email protected]://robotics.ccny.cuny.edu/http://robotics.ccny.cuny.edu/mailto:[email protected]
-
7/30/2019 NUST 1 Introduction
64/102
The City College of New York 64
Outline
Who we are? (Background Information) CUNY City College
CCNY Robotics Lab
PRISM Center @ CCNY
What we are doing? (Some Research Projects)
Wall-climbing robots (ARO, NSF, NCIIA)
Smart brain for miniature robots (NSF MRI)
CAREER project (NSF) Autonomous UAV (ARO)
Education and Outreach Activities
-
7/30/2019 NUST 1 Introduction
65/102
The City College of New York 65
The City College of New York (CCNY)
SUNY (State University of New York)
CUNY (City University of New York)
Stony Brook Albany Buffalo . . .
QueensBrooklyn Graduate CenterCity College . . .Hunter
Founded in 1847 as the Free Academy, CCNY is the flagship campus
ofCUNY system
Eminent CCNY alumni: 9 Nobel Prize Laureates, Colin L. Powell
(formerSecretary of State) , Robert Kahn (co-inventor of the TCP/IP
protocols)
Andrew Grove (Intel co-founder, Donated $26M to School of
Engineering)
1847
()
-
7/30/2019 NUST 1 Introduction
66/102
The City College of New York 66
Grove School of Engineering is the only engineering school in
CUNY system
The site of CUNY Ph.D. Program in Engineering
14 active NSF CAREER Awardees, top among New York state public
universities
CCNY R b ti L b
-
7/30/2019 NUST 1 Introduction
67/102
The City College of New York 67
CCNY Robotics Lab Robotic facility at CCNY
2002 2009US$160,000 start-up US$2.6M research grant science
fellowship, 1 student 3 Post-Docs, 9 Ph.D. students
CCNY Robotics Lab PRISM Center @ CCNY
R b ti T @ CCNY
-
7/30/2019 NUST 1 Introduction
68/102
The City College of New York 68
Robotics Team @ CCNY
Post-Docs (3) Dr. Andrea Gasparri (),
Dr. Ling Guo ()
Dr. Jun-Guo Lu ()
Ph.D. Students (9) Xiaohai Li Ravi Kaushik
Angel Calle
Flavio Cabrera-Mora
Ivan Dryanovski
Narashiman Chakravarthy
Rex Wang
Joseph Samleo
Xiaochen Zhang
Visiting Scholars/Ph.D. students (3): Ronggang Yue ()Song Qiang,
Ying Jiang (Master Students: Wiliam Morris,CUNY Honors College
undergraduate: Igor Labutov
PRISM C t P j t
-
7/30/2019 NUST 1 Introduction
69/102
The City College of New York 69
PRISM Center Project Center forPerceptual Robotics,
Intelligent
Sensors & Machines (PRISM) @ CCNY(Supported by NSF Computing
Research Infrastructureprogram & Investment from Grove School
of Engineering)
Objectives: To enhance comprehensive multidisciplinaryresearch
and education infrastructure at CCNY
To foster collaboration to carry out nationally
competitive research in robotics, vision, wirelesscommunication,
human-computer interaction, etc.
To make CCNY a national urban model forminorityeducation in
robotics and related fields.
Sh i th PRISM
-
7/30/2019 NUST 1 Introduction
70/102
The City College of New York 70
Shaping the PRISM People
PI: Prof. Jizhong Xiao, (CCNY Robotics Lab) Prof. Zhigang Zhu,
(CCNY Visual Computing Lab) Prof. Myung J. Lee, (Advanced
Networking Lab) Prof. George Wolberg, (Vision and Graphics Lab)
Prof. YingLi Tian, (CCNY Media Lab) Prof. Michael Grossberg,
(Graphics Learning & Smart Sensors Lab)
Prof. Ali Sadegh (ME Center for Advanced Engineering Design)
Research Theme RobotizedIntelligentSEnsorNETworks (RISE-NET) in
3D space
Research Thrusts Distributed Perceptual Robotics (robotized
sensor networks, sensing,
control, and coordination of robot teams in 3D space) Multimodal
Intelligent Sensors (multimodal sensors for target
recognition, subject tracking, and event understanding)
Pervasive Smart Machines (human-robot interaction,
human-machine
visual and speech interfaces in a large sensor network) Reliable
Communications (wireless sensor/ad hoc networks,
standardization)
S R h P j t
http://www-cs.ccny.cuny.edu/~zhu/http://www-ee.ccny.cuny.edu/newpage/faculty/people/lee.htmlhttp://www-cs.engr.ccny.cuny.edu/~wolberg/http://www-cs.engr.ccny.cuny.edu/~wolberg/http://www-cs.engr.ccny.cuny.edu/~grossberg/http://www-cs.engr.ccny.cuny.edu/~grossberg/http://www-cs.engr.ccny.cuny.edu/~wolberg/http://www-cs.engr.ccny.cuny.edu/~wolberg/http://www-ee.ccny.cuny.edu/newpage/faculty/people/lee.htmlhttp://www-cs.ccny.cuny.edu/~zhu/
-
7/30/2019 NUST 1 Introduction
71/102
The City College of New York 71
Some Research Projects Wall-Climbing Projects
Funding Agencies: NSF, ARO, NCIIA Smart Brain Project
Funding Agency: NSF MRI
CAREER Project Funding Agency: NSF CAREER
Autonomous Rotorcraft Project Funding Agency: ARO
Swarm Robotics project
Other Projects
Bio-mimetic approaches to walking robot balancing Intelligent
Ground Vehicle Competition (IGVC) Shape Memory Alloy (SMA)
W ll li bi R b t P j t
-
7/30/2019 NUST 1 Introduction
72/102
The City College of New York 72
Wall-climbing Robot Project Project Title:
Cooperative Wall-climbing Robots in 3DEnvironments for
Surveillance and TargetTracking
Objectives: To develop a modular, re-configurable, wall-
climbing robotic system
To investigate intelligent control methodsand vision algorithms
to control and
coordinate a team of such robots to performvarious defense,
security, and inspectionmissions.
M ti ti & A li ti
-
7/30/2019 NUST 1 Introduction
73/102
The City College of New York 73
Motivations & Applications Motivations
Transform the present 2-Dworld of mobile rovers into anew 3-D
universe.
To perform tasks not able byground robots
Applications Building Inspection
New York City law mandatesthe inspection of building
facades every 5 years Infrastructure Inspection
Aircraft Inspection
Urban warfare application
Manual Inspection of buildingfacades: 2 workers and 1
engineerriding on suspended scaffold
E i ti T h l i d R b t
-
7/30/2019 NUST 1 Introduction
74/102
The City College of New York 74
Existing Technologies and Robots Existing wall-climbers:
MSU Flipper & CrawlerJPL-Stanford rock climber
Avionic Instruments Inc.
Vortex attraction techniqueStanford Stickybot
CMU
geckoinspired
climber
Existing Technologies
Cit Cli b Vid
-
7/30/2019 NUST 1 Introduction
75/102
The City College of New York 75
City-Climber Video
The ICRA2006 Best Video Award Finalist, operation on rough &
smooth surfaces,Payload: 4.2kg, US patent No. 7,520,356,
Downloadable: http://robotics.ccny.cuny.edu
S t B i P j t
-
7/30/2019 NUST 1 Introduction
76/102
The City College of New York 76
Smart Brain Project
Project Title: Smart Re-configureable Miniature Robot
Systems Based on System on ProgrammableChip (SoPC)
Technology
NSF MRI Instrument Development
Objectives: to develop highly-adaptive computation module
based on SoPC technology (FPGA) for ultra-
small robots to realize onboard sensor processing, advanced
motion control, and reliable wirelesscommunication
FPGA b d M lti
-
7/30/2019 NUST 1 Introduction
77/102
The City College of New York 77
FPGA-based Multiprocessor
On-Chip Peripheral BUS (OPB)
PLB
Arbiter
ProcessorLoca
lBUS(
PLB)
OPB
Arbiter
PLB to OPB
Bridge
PLB Block
RAM
Interface
Controler
PLB BlockRAM
Interface
Controler
IIC Bus
Interface
Discrete Cosine
Transform
(2D DCT V2.0)
Fast Fourier
Transform
(FFT V2.0)
PLB
Arbiter
PLB to OPBBridge
On-Chip Peripheral BUS (OPB)OPB
Arbiter
IIC Bus
InterfaceGPIO
SDRAM
ControllerOPB
Timer/
Counter
PLB Block
RAMInterface
Controler
ProcessorLocalBUS(
PLB)
DMA
Controller
Dual Port
BRAM
(V5.0)1A0
1A16
2A0
2A16
2D0
2D15
1D0
1D15
2OE
1WE 2WE
1OE
Binary
Counter(V6.0)
CLK
ACLR
Q0
Q16
PowerPC
405 Core
PowerPC
405 Core
PWM
Quadrature
Logic
Motor
PowerAmplifier
Phase A
Phase B
PWM
Logic
Color Camera
Module
SDA
SCL
Y0-Y7UV0-UV7
HREFPCLK
VSYNC
16
17
8
8
Virtex-II Pro
FPGA
UARTRF
Module
Controller
Encoder
A/D
Sensors
Auto-ScanCircuitry
C2I
OPB
Timer/
Counter
CAPT0 CAPT1
INT
Dual Port
BRAM(V5.0)1A0
1A16
2A0
2A16
2D0
2D15
1D0
1D15
2OE
1WE 2WE
1OE
EncoderCounter
Single board FPGA-based
multiprocessor for roboticsapplications(Xilinx Virtex-II Pro
family FPGA)
Two 32-bit IBM PowerPC 405 cores FPGA fabric is used for custom
logicand interfaces.
To build robotics IP core library(Motor control,
wirelesscommunication, & onboard visionprocessing)
Pre-designed, pre-verified, reusablehardware modulesCustomize
hardware circuit in a la
carte fashion
-
7/30/2019 NUST 1 Introduction
78/102
The City College of New York 78
Advancing Mobile Robots to 3D
CAREER Project
-
7/30/2019 NUST 1 Introduction
79/102
The City College of New York 79
CAREER Project
Project Title: Advancing Mobile Robots to 3D
NSF CAREER award
Objectives: to develop a theoretic framework for the planning
and
control of a group of heterogeneous ground robots
&wall-climbing robots operating in the constrained 3Dspace of
urban environments
to develop & experimentally validate algorithms formany
canonical tasks (e.g., 3D map construction,surveillance coverage,
target tracking) employing wall-climbing robots.
Motivations
-
7/30/2019 NUST 1 Introduction
80/102
The City College of New York 80
Motivations
3D map construction
Wall-climbing robot operates
in a 3D constraint space Bird-eye view Avoid occlusion
How to construct 3D map
using multi-modal sensors
How to represent the 3Denvironment efficiently
How to coordinate the multi-robot team
3D Map Construction
-
7/30/2019 NUST 1 Introduction
81/102
The City College of New York 81
3D Map Construction
3D Simulator : Scenario: 3 Ground robots + 1 City-Climber
robotEach robot has a camera and laser scanner (Hokoyu)
Overview
-
7/30/2019 NUST 1 Introduction
82/102
The City College of New York 82
Overview
1. Intra-robot localization:Determine the geometricrelationship
among 4 robotsin 3D space using cameras
Ground robots localization
2. Map integration:
Maps generated by 4 rotary laser rangesensors are integrated
together toconstruct a complete 3D map
Climbing robot localization
Perspective 3 point (P3P)problem
3D Map Construction (Static)
-
7/30/2019 NUST 1 Introduction
83/102
The City College of New York 83
3D Map Construction (Static)
3D laser scanner
Scan coverageStatic robots mappingan corridor
Procedures:
Ground Robots Localization Wall-climbing RobotLocalization (P3P
problem) Rotary laser scan to generateindividual laser point
cloudmap.
3D Laser map fusion ICP algorithm with good
initialestimation
Advantages: Guaranteed convergence of
ICP algorithm No need for laser/cameracalibration Fast speed
Improved accuracy
Experiment Results
-
7/30/2019 NUST 1 Introduction
84/102
The City College of New York 84
Experiment Results
3D Mapping in Indoor Environment
-
7/30/2019 NUST 1 Introduction
85/102
The City College of New York
3D Mapping in Indoor Environment
Scenario
85
The wall climbing robot (WR) and the ground robot (GR) move
intandem and step alternately after pose estimation in
thestationary phase
3D map construction(D i )
-
7/30/2019 NUST 1 Introduction
86/102
The City College of New York
(Dynamic)
86
Experimental Results
-
7/30/2019 NUST 1 Introduction
87/102
The City College of New York
Experimental Results
87
Autonomous Rotorcraft Project
-
7/30/2019 NUST 1 Introduction
88/102
The City College of New York 88
Autonomous Rotorcraft Project Project Title:
Toward Autonomous Miniature Rotorcrafts inCluttered Environments
for Scene Understanding
Objectives: To develop control and navigation algorithms for
small scale rotorcrafts to autonomously explorecluttered and
obstacle-dense environments viamulti-modal sensing
To develop innovative approaches for target
detection and 3D scene understanding usingspatiotemporal image
analysis
To develop a miniature rotorcraft experimentalplatform to test
and verify the proposed methods
Application Scenarios
-
7/30/2019 NUST 1 Introduction
89/102
The City College of New York
Application Scenarios
Surveillance and reconnaissance in urban or woodedenvironments
to enhance tactical situational awareness
In cave search applications to build a 3D map.
Homeland Security: Close-look surveillance, crowdmonitoring,
dynamic placement of camera networks
89
Fig. 1 Miniature UAVs can be used in a) cave search, b)
navigation in wooded
environment, and c) urban warfare.
Miniature Rotorcraft UAVs
-
7/30/2019 NUST 1 Introduction
90/102
The City College of New York
Miniature Rotorcraft UAVs
90
Mobile cameras, VTOL,stable hover
GPS waypoint navigation,
German UAV France Parrot
Dragonfly X-6 Ascending Technologies
CCNY Rotorcraft UAVs
-
7/30/2019 NUST 1 Introduction
91/102
The City College of New York
CCNY Rotorcraft UAVs
91
CCNY ChupaCopter UAV Sensor Suite Configurations
ChupaCopter roll control
City-Flyer
Single Camera Omni-stereo vision
-
7/30/2019 NUST 1 Introduction
92/102
The City College of New York
Single Camera Omni-stereo vision
92
POV-raySimulation
Single Camera Omni-stereo vision
-
7/30/2019 NUST 1 Introduction
93/102
The City College of New York
Single Camera Omni-stereo vision
93
Lin Guo, Igor Labutov, Jizhong Xiao, Design and
calibration of single-camera catadioptric omnistereo
system for Miniature Aerial Vehicles (MAVs),
IROS2010 to appear.
Design optimization of the omni-stereo system for MAVs
Projective model is derived
Sphere-based calibration algorithm
Determine camera parameters and mirror parameters
Minimum of three spheres are needed for calibratation
Algorithms to unwrap the images and derive 3D information
Near Spherical Probabilistic Range Panoramas
-
7/30/2019 NUST 1 Introduction
94/102
The City College of New York
Near Spherical Probabilistic Range Panoramas
94
Igor Labutov, Carlos Jaramilos, Jizhong Xiao, Generating
Near-Spherical Probabilistic
Range Panoramas Using a Low-Cost, Single-Camera
Catadioptric-Stereo Rig, IROS2010
Best presentation award, Omnivis2010, Robotics: Science and
Systems, June 23~26, 2010
Stereo + Optical Flow
How can we get the full sphere of depth?
3D SLAM Research
-
7/30/2019 NUST 1 Introduction
95/102
The City College of New York
3D SLAM Research
95
Simultaneous Localization and Mapping in 3D space
Multi-volume Mapping
Outline
-
7/30/2019 NUST 1 Introduction
96/102
The City College of New York 96
Outline
Who we are? (Background Information) CUNY City College
CCNY Robotics Lab
PRISM Center @ CCNY
What we are doing? (Some Research Projects)
Wall-climbing robots (ARO, NSF, NCIIA)
Smart brain for miniature robots (NSF MRI)
CAREER project (NSF)
Autonomous UAV (ARO)
Swarm robotics project
Education and Outreach Activities
Robotics Outreach Activities
-
7/30/2019 NUST 1 Introduction
97/102
The City College of New York 97
Robotics Outreach Activities
CCNY Autonomous Vehicle Design Club
CCNY Robotics Club
IGVC-CCNY 2007
http://www.igvc-ccny.org/
http://ccnyrobotics.org/
Education & Outreach Activities
-
7/30/2019 NUST 1 Introduction
98/102
The City College of New York 981st, 2nd places in IEEE
Micromouse
Competition 2010
CCNY Robotics Club Intelligent Ground Vehicle Competition
(IGVC)
Senior Design Projects Research Experience for Undergraduates
FIRST Robotics Competition for HS teams Summer Internship
Program
Best student paper award
CCNY IGVC Teams
-
7/30/2019 NUST 1 Introduction
99/102
The City College of New York 99
IGVC2010, Team City Alien
1st place in design competition
IGVC2009, 4th place in Autonomous Challenge
CCNY IGVC2007
CCNY IGVC2008
IGVC2010, Team CAP10
CCNY IGVC Videos
-
7/30/2019 NUST 1 Introduction
100/102
The City College of New York 100
1st place in design competition, IGVC2010
4th place in Autonomous Challenge, IGVC2009
Robotics Outreach Activities
-
7/30/2019 NUST 1 Introduction
101/102
The City College of New York 101
Robotics Outreach Activities CCNY Robotics Club
FIRST roboticscompetition
Thank you!
-
7/30/2019 NUST 1 Introduction
102/102
Thank you!
