Vision-Based Motion Control of Robots Azad Shademan Guest Lecturer CMPUT 412 – Experimental...

Vision-Based Motion Control of Vision-Based Motion Control of RobotsRobots

Azad ShademanGuest Lecturer

CMPUT 412 – Experimental RoboticsComputing Science, University of Alberta

Edmonton, Alberta, CANADA

A. Shademan. CMPUT 412, Vision-based motion control of robots March 4, 2008

Vision-Based Control

current

desired

Left Image Right Image

Vision-Based Control

Left Image Right Image

Vision-Based Control Feedback from visual sensor (camera) to

control a robot Also called “Visual Servoing” Is it any difficult?

Images are 2D, the robot workspace is 3D 2D data 3D geometry

Where is the camera located? Eye-to-Hand

e.g.,hand/eye coordination

Eye-in-Hand

Visual Servo Control law Position-Based:

Robust and real-time pose estimation + robot’s world-space (Cartesian) controller

Image-Based:Desired image features seen from cameraControl law entirely based on image features

Position-Based

Desired pose

Estimated pose

Image-Based

Desired Image feature

Extracted image feature

Visual-motor Equationx1

q=[q1 … q6]

Visual-Motor Equation

This Jacobian is important for motion control.

Visual-motor JacobianImage spacevelocity

Joint spacevelocity

Image-Based Control Law

1. Measure the error in image space

2. Calculate/Estimate the inverse Jacobian

3. Update new joint values

Image-Based Control Law

Desired Image feature

Extracted image feature

Jacobian calculation Analytic form available if model is known.

Known model Calibrated

Must be estimated if model is not known

Unknown model Uncalibrated

Image Jacobian (calibrated) Analytic form depends on depth estimates.

Camera/Robot transform required. No flexibility.

CameraVelocity

Image Jacobian (uncalibrated) A popular local estimator:

Recursive secant method (Broyden update):

Relaxed model assumptions

Traditionally: Local methods No global planning Difficult to show

asymptotic stability condition is ensured

The problem of traditional methods is the locality.

Calibrated vs. Uncalibrated

Model derived analytically Global asymptotic

stability Optimal planning is

possible A lot of prior knowledge

on the model

Global Model Estimation (Research result)

Optimal trajectory planning

Global stability guarantee

Synopsis of Global Visual Servoing Model Estimation (Uncalibrated) Visual-Motor Kinematics Model Global Model

Extending Linear Estimation (Visual-Motor Jacobian) to Nonlinear Estimation

Our contributions:K-NN Regression-Based EstimationLocally Least Squares Estimation

Local vs. Global

Key idea: using only the previous estimation to estimate the Jacobian

1. RLS with forgetting factor Hosoda and Asada ’94

2. 1st Rank Broyden update: Jägersand et al. ’97

3. Exploratory motion: Sutanto et al. ‘98

4. Quasi-Newton Jacobian estimation of moving object: Piepmeier et al. ‘04

Key idea: using all of the interaction history to estimate the Jacobian

Globally-Stable controller design

Optimal path planning Local methods don’t!

K-NN Regression-based Method

KNN(q)

Locally Least Squares Method

Experimental Setup

Puma 560 Eye-to-hand configuration Stereo vision Features: projection of the end-effector’s

position on image planes (4-dim) 3 DOF for control

Measuring the Estimation Error

Global Estimation Error

Noise on Estimation Quality

With increasing noise level, the error decreases

Effect of Number of Neighbors

Conclusions Presented two global methods to learn the visual-

motor function LLS (global) works better than the KNN (global) and

local updates. KNN suffers from the bias in local estimations Noise helps system identification

Eye-in-Hand Simulator

Mean-Squared-Error

Task Errors

Questions?

Position-Based Robust and real-time relative pose

estimation Extended Kalman Filter to solve the

nonlinear relative pose equations. Cons:

EKF is not the optimal estimator.Performance and the convergence of pose

estimates are highly sensitive to EKF parameters.

Overview of PBVS

2D-3D nonlinear point correspondences

IEKFWhat kind of nonlinearity?

T. Lefebvre et al. “Kalman Filters for Nonlinear Systems: A Comparison of Performance,” Intl. J. of Control, vol.

77, no. 7, pp. 639-653, May 2004.

EKF Pose Estimation

Measurement noise

State variable

Process noise

yaw pitch roll

Measurement equation is nonlinear and must be linearized.

Visual-Servoing Based on the Estimated Global Model

Control Based on Local Models

In practice: Broyden 1st-rank estimation, RLS with forgetting factor, etc.

Estimation for Local Methods

Vision-Based Motion Control of Robots Azad Shademan Guest Lecturer CMPUT 412 – Experimental...

Documents

Azad Details

CMPUT 301: Lecture 25 Graphic Design

azad kumar

The University of Azad Jammu and Kashmir - Muzaffarabad Azad

Nigaarishaat e Azad - Maulana Abul Kalam Azad

Azad Foundation

CMPUT 651: Front to Front Perimeter Search & Pattern Databases

CMPUT 412 3D Computer Vision Presented by Azad Shademan Feb. 13-15, 2007

Knowdiff visiting lecturer 140 (Azad Shademan): Uncalibrated Image-Based Robotic Visual Servoing

Tanziyaat e Azad - Maulana Abul Kalam Azad

CMPUT 391 – Database Management Systems Department of Computing Science University of Alberta CMPUT 391 Database Management Systems JavaServer Pages (JSP)

CMPUT 412 Actuation

CMPUT 114 – First Class C. Jones, Winter 2003Slide # 1 CMPUT 114 – Welcome! Department of Computing Science University of Alberta

English A: language and literature – Higher leel – … PAST PAPERS - YEAR/2018...Supervised autonomous robotic soft tissue surgery, Azad Shademan, Ryan S. Decker, Jusin D. Opfermann,

Computing Science (CMPUT) 455

CMPUT 412 Binary Image Processing

Azad Ki Taqreerain - Maulana Abul Kalam Azad

Introduction to Vision-Based Control or Robotscomputerrobotvision.org/2009/tutorial_day/crv09_visual_servo.pdfAzad Shademan (a.shademan@ieee.org), Introduction to Vision-Based Motion

CMPUT 366: Intelligent Systems and ... - web.stanford.eduweb.stanford.edu/class/cme241/lecture_slides/rich...CMPUT 609: Reinforcement Learning & Artiﬁcial Intelligence Instructors:

AZAD Slideshow