Upload
chester-price
View
215
Download
0
Embed Size (px)
Citation preview
NA-MICNational Alliance for Medical Image Computing http://na-mic.org
OpenIGTLink Update
NA-MIC Summer Project Week 2010
Junichi Tokuda, Ph.D.
Department of Radiology
Brigham and Women’s Hospital
NCIGTNational Center for Image Guided Therapy
http://ncigt.org
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Backgrounds
• Hardware/software in IGT– Position tracking sensors– MR/CT/Ultrasound scanners– Robotic devices– Planning and navigation software
• Data types– Images– Positions / Transforms– Software / hardware status, etc…
• Communication standard for IGT?
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Goals of OpenIGTLink
• Available – Research and commercial
• Simple– From embedded system to HPC
• Extensible– Variety of data types
• Reliable– Used clinically!
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Protocol − Overview
• Message-based Protocol– No session / messages are independent– Allows application-specific message types
Header
IMAGE Type Body
Body
TRANSFORM Type Body
COMMAND Type Body
….
Body Size
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Protocol − Body
• Position / Transform
x y z ox oy oz w
position orientation (quaternion)
tx ty tz sx sy sz nx ny nz x y z
€
tx sx nx xty sy ny ytz sz nz z0 0 0 1
⎛
⎝
⎜ ⎜ ⎜
⎞
⎠
⎟ ⎟ ⎟
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Protocol − Body
• Image – 2D/3D– Integer / floating point– Scalar / vector– Partial Image update i
j
k
Nj
(0, 0, 0)
(Ni -1, Nj -1, Nk -1)Nk
Mj
Ni
Mi
(oi +Mi -1, oj +Mj -1,ok+Mk -1)
Mk
(oi, oj, ok)
Generic info size transform partial volumeindex/size
Pixel data
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
The OpenIGTLink Library
• Multi-platform C/C++ Library– Windows, Linux, Mac OS X, Solaris– Used in QNX, iPhone OS, vxWorks etc..
C++ socket & thread class C message structure
C++ message class
Application
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
The OpenIGTLink Library
• Code snippet (9 lines to send position)
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
The OpenIGTLink Library
• Various “hello world” examples– Tracker simulator– Imager simulator– Data receive server– etc.
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
• Import to / export from MRML scene– Image (vtkMRMLScalarVolumeNode)– Transform (vtkMRMLLinearTransformNode)
3D Slicer OpenIGTLink I/F
Tracker
OpenIGTLink
Imager
OpenIGTLink
…
3D Slicer
Connector node 1
Connector node 2
Ext. software
OpenIGTLink
Connector node N
MRML Scene
Module 1
Module 2
Module N
Image
LinearTransform
Image
LinearTransform
Ethernet
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
3D Slicer OpenIGTLink I/F
• Import / export
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
3D Slicer + BrainLAB
Elhawary H, Norton I, Patel P, Rigolo L, Papademetris X, Hata N, Golby AJ (BWH / Yale / BrainLAB)
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
3D Slicer + BrainLAB
• Research prototype (Slicer) +
FDA-approved IGS system (BrainLAB)
3D Slicer
OpenIGTLink
BioImage Suit
OpenIGTLVVLink
VVLink
BrainLABVectorVision
Image / Tracking data
(VVLink)
Image / Tracking data
(OpenIGTLink)
Research PrototypeIGS system
Bridge Software
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
MRg Prostate Biopsy Robot
Fischer G, Tokuda J, Song S, Iordachita I, DiMaio S, Cho N, Csoma C,Bleivns J, Burdette EC, Fichtinger G, Hata N, Tempany C
(Johns Hopkins, BWH, Queen’s, Acoustic MedSystems)
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
MRg Prostate Biopsy Robot
• Integration of Slicer, robot and scanner
3D Slicer
OpenIGTLink
Robot Proxy
OpenIGTLink Pipe
Controller
Pipe
Scanner Proxy
OpenIGTLink
MRI Scanner
ProprietaryProprietary
Real-time image
Imaging slice position
Current position
Command
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Neurosurgical Manipulator
Arata J, Tada Y, Yamada A, Tanaka Y, Fujimoto H(Nagoya Institute of Technology, Japan)
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Neurosurgical Manipulator
• System diagram
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Motion Correction for MRgFUS
• MR Image Motion Correction by US-based motion tracking
Schwartz BM, McDannold NBrigham and Women’s Hospital
MRI Scanner
No Motion w/ motionUncorrected
w/ motionCorrected
ScannerHardware Imaging slice position
OpenIGTLink
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Cardiac EP Catheter + 3D Slicer
Jagadeesan J, Hata NBrigham and Women’s Hospital
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Cardiac EP Catheter + 3D Slicer
• Matab/Simulink in backstage…
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Next Steps
• OpenIGTLink Protocol Version 2 (draft)– Backward compatibility with version 1
– Query support
– Better IGS system support
– Other message types?
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Query Support
• “Get” message to request data
• “Start”, “stop” and “pause” messages to control data streaming
• “Status” message to respond to query
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
IGS Systems Support
• Remote Image List– Display a list of images available in the
remote software
• Tracking data– Multi-channel, device name
• Fiducial, trajectory– Share surgical planning information
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Other Message Types?
• Remote Execution– Run Tcl/Python on 3D Slicer from remote– Remote procedure call
• XML data / Associative array– General data types– Versatility but compromised performance
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
MATLAB Interface
• MATLAB – 3D Slicer communication
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
OpenIGTLink + ZeroMQ
• A layer between OpenIGTLink and TCP/IP• Support different message patterns
– Point-to-point– Publish-subscribe– Request-reply– Parallelized pipeline
• Proposed in CTKhttp://www.commontk.org/index.php/Documentation/Messagingby Patrick Cheng (Georgetown)
Ethernet
IP
TCP
ZeroMQ
OpenIGTLink
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Acknowledgements
• BWH. Nobuhiko Hata, Ron Kikinis, Clare Tempany, Alexandra Golby, Haiying Liu, Tina Kapur, Jayender Jagadeesan, Haytham Elhawary, Benjamin M. Schwartz, …
• Isomics, Inc. Steve Pieper• Kitware, Inc. Luis Ibanez• JHU. Sam S. Song, Nathan Cho, Iulian Iordachita, Peter Kazanzides, …• Queen’s. Univ. Gabor Fichitinger• Georgetown Univ. Ziv Yaniv, Patrick Cheng, Kevin Cleary• Acoustic MedSystems Inc. Jack Blevins, E. Clif Burdette• Worcester Polytechnic Institute. Gregory Fischer• AIST, Japan. Kiyoyuki Chinzei• Nagoya Inst of Tech. Junpei Arata, Atsushi Yamada, Hiroaki Kozuka, Hideo
Fujimoto• Nagoya Univ. Yuichiro Hayashi
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Grant Support
• This work is supported by 1R01CA111288, 5U41RR019703, 5P01CA067165, 1R01CA124377, 5P41RR013218, 5U54EB005149, 5R01CA109246, R01EB006494 and R21EB007770 from the NIH. This study is also in part supported by NSF 9731748, CIMIT, Intelligent Surgical Instruments Project of METI (Japan).
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
Publications
• Tokuda J., Fischer G.S., Papademetris X., Yaniv Z., Ibanez L., Cheng P., Liu H., Blevins J., Arata J., Golby A.J., Kapur T., Pieper S., Burdette E.C., Fichtinger G., Tempany C.M., Hata N. OpenIGTLink: an open network protocol for image-guided therapy environment. Int J Med Robot. 2009 Dec;5(4):423-34. PMID: 19621334. PMCID: PMC2811069.
• Tokuda J., Fischer G.S., DiMaio S.P., Gobbi D.G., Csoma C., Mewes P.W., Fichtinger G., Tempany C.M., Hata N. Integrated navigation and control software system for MRI-guided robotic prostate interventions. Comput Med Imaging Graph. 2010 Jan;34(1):3-8. PMID: 19699057. PMCID: PMC2815337.
• Arata J., Kozuka H., Kim H.W., Takesue N., Vladimirov B., Sakaguchi M., Tokuda J., Hata N., Chinzei K., Fujimoto H. Open core control software for surgical robots. Int J Comput Assist Radiol Surg. 2010 May;5(3):211-20. PMID: 20033506.
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
For more information…
OpenIGTLink ML: [email protected]
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
History
• Dec 2007: Discussed in IGT Week• Jan 2008: Defined in All Hands Meeting• May2009 : Supported in 3D Slicer 3.4• Jun 2010: Protocol Extension
National Alliance for Medical Image Computing http://na-mic.org
National Center for Image Guided Therapyhttp://ncigt.org
3D Slicer OpenIGTLink I/F
• Performance Optimization by Steve
Before After