Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Robot registration
Jocelyne TROCCAZTIMC [email protected]://www-timc.imag.fr/gmcaohttp://www-timc.imag.fr/Jocelyne.Troccaz
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Grenoble GMCAO (CAMI) teamCreated in 1985 by Philippe CinquinHeaded from 1996 by Jocelyne TroccazStrong connection to Grenoble HospitalAbout 35 people
Grenoble HospitalWe are thereGrenoble
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Research topics
Imaging
Modelling
Planning and simulation
Data fusionRobotics
Navigation
Sensors
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
What is it?Registration consists in determining geometric relationships between two reference framesRobot registration essentially consists in transferring the planning to the robot coordinate system
Pre-operative data
Intra-operative data
Robot
Image registration
?
?
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Example
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Tools
CalibrationTrackingData registration
Using:Patients’ dataExternal objects
RequiresIntrinsic robot calibration
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Contents
IntroductionMethodsExamples: four main situationsConclusion
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Possible reference frames of interest
Rpatient
planningRimg_sensor(i)
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rlocalizer
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Hardware examplesLocalizers:
Optical, US, magnetic, mechanical arm
Imaging sensors:Fluoroscopy, digital X-Ray, ultrasound
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Examples A: Robodoc, ACRobot, (CAD-Implant)
Rpatient
planning
Intra-operativelyPre-operatively
Rpatient
Rrobot
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Examples B: CASPAR
Rpatient
planning
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rlocalizer
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Examples C:Speedy, Cyberknife, MARS
Rpatient
planning Rimg_sensor1
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rimg-sensor2
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Example D: Cyberknife+Synchrony
Rpatient
planningRimg_sensor1
Intra-operativelyPre-operatively
RpatientRrobot
Rlocalizer
Rimg_sensor2
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Registration basics
Two reference frames RA and RB and a transform TA
B to be determinedSelection of features FA in RA and FB in RB
Definition of a similarity measure (or distance) between FA and FB
Determination of TAB such that the
similarity is maximum (or distance minimum)
TAB= arg min d(FA,TA
B(FB))
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Typical 3D/3D rigid registration methods
Point to point (Procrustes)
External fiducialsAnatomical landmarks
Surface registrationAnatomical surface (i.e. ICP, chamfer matching)Template [Radermacher]
Intensity-basedregistration (for images only)
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Examples APre-op: planning on CT dataIntra-op: a robotDeveloped methods:
Robodoc: robot palpation of implantedfiducialsACRobot: robot palpation of anatomicalsurfaceCAD-Implant: fiducials+template (robot is pre-operative)
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Robodoc (for hip surgery)
Define precisely the prosthesis position (geometrical or biomechanicalcriteria)Improve the preparation of the hip cavity
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Robodoc [Taylor et al.]
1. Planning: Orthodoc
3. Intra-operative bone milling procedureusing Robodoc (based on the IBM scara robot)
2. Pre-op to intra-op registrationusing implanted titanium pins (anatomical registration in the last version)
6D force sensor
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Example A.1: Robodoc
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rpatient
planning
3D/3D point-to-pointregistration
Palpation
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
ACRobot [Davies et al.]
« Hands on » robotKnee arthroplasty3 DOFsBone surface palpated with the robotIEEE TRA 03: registration accuracy evaluation
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
ACRobot registration tests
Initial estimate from 4 anatomical landmarksICP surface matchingIntra-op criterion: rms distanceDoes a small rms mean a good registration?Experiments with phantom and artificialdata:
Generated palpated points (10 to 100) with or withoutrandom noise added (max up to 1.5mm)Initial estimates in the range of +/-10mm and 2°Known translational and rotational errors
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
ACRobot: results
Nb pts <70 makesICP more problematicResults highlydepend on the data setsMay have a rms=0.6 witherrors of 0.8mm and 2° From [IEEE-TRA03]
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Example A.2: ACRobot
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rpatient
planning
ICP 3D/3D surfaceregistration
Palpation
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
CAD-Implant [Champleboux et al.]
A system for dental implant assistanceA template associated to fiducials visible on CTA pre-operative robotIntra-operatively: no robot, no computersSurface registration withoutcomputers
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Example A.3: CAD-Implant
Rpatient
planning
Intra-operativelyPre-operatively
Rpatient
Rrobot
Line-to-line registration
calibration
Template matching
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Example B
Close to RobodocKnee application Pre-op: planning on CT dataIntra-op: a robot, a tracking sensorDeveloped method:
implanted fiducials S for registrationfiducials S’ for motion detection
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Example B: CASPAR
Rpatient
planning
Intra-operativelyPre-operatively
Rpatient Rrobot
3D/3D point-to-pointregistration
Palpation
Rlocalizer
Motion detection
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Examples C
Pre-op: planning on CT dataIntra-op: a robot, X-Ray sensorsDeveloped methods:
Speedy V1 [Lavallée89]: Direct X-ray/robot calibration and manual image registrationCyberknife V1 [Schweikard98]: Indirect X-ray/robot calibration and intensity-based registration
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Speedy V1
Stereotactic neurosurgeryPre-operative MR or CTIntra-operative X-Ray (AP and lateral) – several examsDirect X-Ray/robot calibrationManual image registration (anatomical for pre-op/intra-op and markers for intra-op/intra-op)
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
SpeedyX-Ray extrinsiccalibration: bi-plane model
2D/2D image registration fiducials
source
(P1) (P2)
MM2M1
f1: M (u,v) M1(x1,y1)f2: M (u,v) M2(x2,y2)
image
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Example C.1: Speedy V1
Rpatient
planning Rimg_sensor1
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rimg-sensor2
X-ray/robotcalibration
X-Ray calibration
imageregistration
intra-operativeimage registration
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Cyberknife V1 [Schweikard et al.]
RadiotherapyapplicationComplextrajectories for improved tumordestruction (multiple radiation ports)6 DOFs requiredVery heavy tools
Traditional linearaccelerator set-up
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Cyberknife V1
Planning on CT dataIntra-operatively:
Indirect X-Ray/robot calibration (via isocenter)« X-Ray/pre-computedDRRs » intensity-basedregistration (before eachbeam activation) Small motion compensation when necessary / interruption of the procedureand replanning for large motion
A Digitally ReconstructedRadiograph (DRR)
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Example C.2: Cyberknife V1
Rpatient
planning Rimg_sensor1
Intra-operativelyPre-operatively
Rpatient
Rrobot
Rimg-sensor2
Robot/Isocalibration
X-Ray calibration
iterated imageregistration
Risocenter
X-Ray/Iso calibration
Set-up
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Example D: Cyberknife+synchronyPre-op: planning on CT dataIntra-op: a robot, two X-ray sensors, a localizerDeveloped methods:
X-Ray/robot calibrationX-Ray/DRR registration for head motion compensationOr fiducial-basedregistration plus real-time tracking for targetsmoving with respiration
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Real-time registration
Large motion tracking[Schweikard05]
Internal fiducials (gold seeds) for initial registration External fiducials (IR diodes) for respiration trackingLearning internal/externalfiducials relationship
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Example D: Cyberknife+Synchrony
Rpatient
planning Rimg_sensor1
Intra-operativelyPre-operatively
Rpatient=Riso
Rrobot
Rlocalizer
Rimg_sensor2
Pre-learning
tracking
imagecalibration
X-Ray/robot calib.
initialregistration
indirect
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Another type of solution (E)Target defined in the intra-operative imaging data*Examples
Indirect visual servoing: computing the robot position from the images
PAKY+RCM [Stoianovici et al.], LPR [Cinquin et al.], etc.
Direct visual servoing: modeling variations of the robot position to variations of the target in the images
GABIE [Morel], ZEUS [deMathelin], etc.
*if pre-operative planning: need for pre-op/intra-op registration
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Example E.1
planning
Rimg_sensor=Rpatient
Intra-operatively
Rrobot
Image analysisand robot model
PAKY+RCM
LPR
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Example E.2
planning
Rimg_sensor=Rpatient
Intra-operatively
Rrobot
Image analysis andImage/robot differential model
toolheart
US plane(ω1,ω2,ω3,d)=f(d(T1,P1), d(T2,P2))
T1
T2
P1
P2
GABIE
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
DiscussionModus operandi:many solutions
Palpation (fiducial, anatomy): easy, invasiveImaging (anatomy): more difficult, less or non invasiveTemplate: easy, limited to few applicationsNeed for updated or real-time registration?
No motionMotion detectionDiscrete motion detection and compensationContinuous motion detection and tracking
Summer School on Surgical Robotics Montpellier, Sept. 07
Robot Registration – J. Troccaz
Discussion (cont’d)No universal recipies: depends on the applicationSome important issues
Intra-operative evaluation of registration accuracySafety of real-time registration