A marker-based contactless catheter-sensing method to detectsurgeons’ operations for catheterization training systems

Jin Guo1& Shuxiang Guo2,3

& Maoxun Li1 & Takashi Tamiya4

# Springer Science+Business Media, LLC, part of Springer Nature 2018

AbstractIt is challenging to position a catheter or a guidewire within a patient’s complicated and delicate vascular structure due to the lack ofintuitive visual feedback by only manipulating the proximal part of the surgical instruments. Training is therefore critical before anactual surgery because any mistake due to the surgeon’s inexperience can be fatal for the patient. The catheter manipulation skills ofexperienced surgeons can be useful as input for training novice surgeons. However, few research groups focused on designs withconsideration of the contactless catheter motion measurement, which allows obtaining expert surgeons’ catheter manipulationtrajectories whilst still allowing them to employ an actual catheter and apply conventional pull, push and twist of the catheter asused in bedside intravascular interventional surgeries. In this paper, a novel contactless catheter-sensing method is proposed tomeasure the catheter motions by detecting and tracking a passive marker with four feature-point groups. The passive marker isdesigned to allow simultaneously sensing the translational and rotational motions of the input catheter. Finally, the effectiveness ofthe proposed contactless catheter-sensing method is validated by conducting a series of comparison experiments. The accuracy anderror analysis are quantified based on the absolute error, relative error, mean absolute error, and the success rate of the detection.

Keywords Catheter sensing . Contactless measurement . Image processing . Catheterization training system . Intravascularinterventions

1 Introduction

Minimal damage to healthy tissue, less blood loss and shorterrecuperation time has led to extensive adoption of catheter-

based intravascular interventional surgeries (Rafii-Tari et al.2014). However, surgeons have to face a number of chal-lenges while manipulating a catheter to target a positionwithinthe patients’ intricate and delicate vascular structure. Due tothe lack of intuitive visual feedback, surgeons always carryout the operations by imagining the spatial relationship be-tween the catheter and its surrounding vessels based on theirexperience and the tactile feedback received (Wang et al.2011). Notably, complicated vessel shapes and multi-contacts between the catheter and the blood vessel walls leadto significant decrease in the feasibility and maneuverabilityof positioning the catheter (Wang et al. 2014). The cathetermanipulation skills of experienced surgeons are therefore crit-ical to the success of catheter-based intravascular intervention-al procedures. To provide novice surgeons with the opportu-nity to learn basic catheter manipulation skills, as well asachieve the reduction of risks to both surgeons and patients,both the virtual reality-based and silicone-based simulatorswere used to train inexperienced surgeons before they arecompetent for the real surgeries (Aggarwal et al. 2006; Ikedaet al. 2006) as shown in Fig. 1. The VR-based simulators, withdistinct advantages over conventional training methods oncadavers, animals or real patients due to practical and ethical

* Shuxiang Guoguo@eng.kagawa-u.ac.jp

Jin Guoguojin0522@gmail.com

Maoxun Lis14d505@stmail.eng.kagawa-u.ac.jp

1 Graduate School of Engineering, Kagawa University,Takamatsu, Japan

2 Key Laboratory of Convergence Medical Engineering, System andHealthcare Technology, The Ministry of Industry and InformationTechnology, School of Life Science, Beijing Institute of Technology,No. 5, Zhongguancun South Street, Haidian District, Beijing 100081,China

3 Department of Intelligent Mechanical Systems Engineering, KagawaUniversity, Takamatsu, Kagawa, Japan

4 Department of Neurological, Surgery Faculty of Medicine, KagawaUniversity, Takamatsu, Kagawa, Japan

Biomedical Microdevices (2018) 20:76 https://doi.org/10.1007/s10544-018-0321-5

reasons, can provide not only flexible and repeatable scenariosbut also the possibility to precisely assess training progress(Samur et al. 2012; Chen et al. 2015; Li et al. 2012).Silicone-based vascular simulators, with the three dimensionalvascular structure fabricated based on CT/MRI files with13 μm resolution and similar physical characteristics to realblood vessels, have been developed to provide a practicalenvironment for training and even skill evaluation (Kodamaet al. 2012; Tercero et al. 2010, 2013). The three-dimensionalcatheter manipulation trajectories obtained from experiencedsurgeons can provide valuable instructions for training novicesurgeons. For instance, novice surgeons can improve upontheir catheterization skills by emulating their manipulationtrajectories in order to avoid improper operations that lead todamages in the patients’ blood vessel walls (Kodama et al.2014). Therefore, measuring the catheter motions whilst stillproviding a familiar and ergonomic setting for the experiencedsurgeons and preserving the manual catheter manipulationskills remains a challenge for both training and quantitativeevaluation.

Currently, commercially available interfaces, optical en-coders, electromagnetic sensors and optical mouse sensorsare used to record the experienced surgeons’ catheter manip-ulation skills. The Sensei X andMagellan TM robotic cathetersystems (Saliba et al. 2008; Riga et al. 2013) (HansenMedicalInc., Mountain View, CA, USA) utilized either a 3D joystickor navigation buttons on the workstation to drive the catheter.Additionally, the PHANTOM Omni (Lu et al. 2011) andNintendo Wii (Wei et al. 2009) are employed to manipulateand record the motion of the simulated catheter in the virtualreality environments. Unfortunately, the commercial inter-faces are not manipulated in accordance with conventionalbedside catheterization procedures. In order to allow surgeonsto apply conventional insertion, extraction and rotation mo-tions, Fukuda et al. (Tanimoto et al. 1997; Negoro et al. 2002)have designed and developed a two-joint mechanism in whicheach joint was connected to a motor with an optical rotary

encoder based on a wire mechanism. A similar design wasproposed by the researchers in (Guo et al. 2012; Feng et al.2015). Unlike conventional bedside intravascular interven-tions, surgeons have to manipulate a Bmetal rod^ which hasdifferent size and material attributes with a real catheter, eventhough two optical encoders are capable of recording thecatheter motions. Thakur et al. (2009, 2007) developed a cath-eter sensor to measure the radial and axial motions of the inputcatheter based on two optical encoders, which allows surgeonsto use a real catheter as the controller. The axial motion wasmeasured by the optical encoder through two rollers that me-chanically couple to the catheter while the radial motion wasdetermined by the proposed rotational sensor including threeminiature bearings and an optical encoder disk. Another sim-ilar two-roller catheter sensing mechanism was proposed bythe researchers in (Ma et al. 2013). However, the catheter isclamped by the rollers coupled with encoders to measure themotion of the input catheter. An initial force has to be appliedto the rollers to ensure the input catheter does not slip in thecatheter-sensing unit, which hinders the surgeons’ tactile feed-back in the actual surgeries due to the friction induced by therollers. In Tercero and Ikeda’s implementation (Tercero et al.2007; Shi et al. 2012), the electromagnetic (EM) sensor wasattached to the catheter tip to measure its 3D trajectories, butthis design resulted in the changes on the shape and stiffnessof the catheter tip and led to further difficulties in cathetermanipulation. In addition, the EM tracking systems were sus-ceptible to environmental inferences such as metal objects inor near the tracking area (Brattain et al. 2011), and need morecomplex calibration (Stoll and Dupont 2005). Researchers in(Yan et al. 2010; Guo et al. 2013) employed the opticalmouse sensors to determine the input catheter’s move-ments which are capable of the contactless measurement,but they were limited to the size of the input catheter. Thisdesign will not work especially in the rotational measure-ment when sensing the catheters with smaller sizes.Furthermore, the smooth surface of the catheters could

Fig. 1 The overview of thecatheterization training systemsbased on the virtual realitytechnique and the endovascularevaluator

76 Page 2 of 11 Biomed Microdevices (2018) 20:76

result in bad measurement results due to the lack of thefeature points.

To overcome these problems, we presented the marker-based contactless catheter-sensing method to measure the ra-dial and axial motions of the input catheter simultaneouslywhile still preserving the manual catheter manipulation skillsused in clinical practice. The proposed method to detect thefeature points and determine the catheter motions was de-scribed in detail. The experiments conducted to evaluate theperformance of the proposed catheter-sensing method werepresented.

The rest of this paper is organized as follows: In Section2, the design of the catheter sensor is introduced. In section3, a series of comparison experiments for validating theperformance of the proposed catheter sensor are presented.Finally, the discussion and conclusions are presented inSections 4 and 5.

2 Materials and methods

During the conventional intravascular interventional pro-cedures, surgeons navigate the catheter by two degrees offreedom (pull, push and twist), which can generate a 3Dtrajectory inside the patients’ blood vessels. To obtain theexperienced surgeons’ catheter manipulation trajectories, acontactless catheter-sensing method was presented to allowthe surgeons to apply the conventional catheterizationskills using a real catheter as the controller. The advantagesand the primary contributions of this approach are summa-rized as follows:

1) Compared to the existing catheter sensing methods, theproposed sensing algorithm can realize contactless mea-surement of the catheter manipulation skills therebyavoiding additional friction;

2) The passive marker can be used to measure the radial andaxial motion of the input catheter simultaneously. It canalso be extended to the other applications in which thecontactless measurement is needed;

3) Compared with our previous method (Guo et al. 2016),the distribution of the feature points in the passive markeris redesigned. Critical situation detection and a displace-ment compensation method are introduced to improve thestability of the catheter-sensing algorithm;

4) A plastic hollow cylinder is used to enlarge the size of thesensing part thereby improving the measurement accura-cy especially in the rotational measurement. Moreover,this method can allow sensing the motions of the catheterswith smaller sizes;

5) The sensing principle is simple and can be easily integrat-ed to more complicated surgical systems.

2.1 Overview of the proposed catheter motion sensor

The proposed catheter motion sensor consists of a camera, twocatheter holders, an over-tube, two over-tube holders, and aplastic hollow cylinder with a passive marker (shown inFig. 2). A camera is positioned right above the catheter todetect and track the feature points of the passive marker. Theaxial and radial motions of the input catheter are determinedaccording to the movements of the feature points. The inputcatheter passes through and is coupled with the over-tubewhich is used to avoid the catheter’s bending and to guaranteethat the catheter is straight under the camera. A passive markerincluding four groups of feature points is attached to a plastichollow cylinder which is coaxially aligned with the over-tube.The plastic hollow cylinder is used to enlarge the size of themeasurement part. Surgeons are capable of manipulating thereal catheter directly according to their own habits. The plasticover-tube and the hollow cylinder do not lead to deteriorationof tactile feedback of a catheter since the weights of these twoparts are very light. Additionally, lubricants are applied on thecatheter holders to decrease the frictional resistance duringsurgeons’ operations. The height of the camera can be modi-fied by moving the junctions between the camera holders andthe over-tube holders. The maximum stroke length of thetranslational motion is about 150 mm, and there is no limita-tion in rotational direction.

2.2 Distribution of the feature points in a passivemarker

It is challenging to directly capture the motions of the inputcatheter by camera because the surface of the catheter is toosmooth. A passive marker is therefore designed to make iteasier to compute the translational displacement and rotationalangles of the input catheter. We define four feature points

Fig. 2 The structure of the proposed catheter motion sensor

Biomed Microdevices (2018) 20:76 Page 3 of 11 76

(black points) with two connected lines as a group. The pas-sive marker includes four groups of the feature points (groupA, B, C and D) and the four feature-point groups are alignedconcentrically. Every two black feature points deployed in agroup have a 5-mm-spacing, and the intersection angle be-tween every two neighboring feature-point group is 90°. Thespecific marker distribution is shown in Fig. 3.

2.3 Feature points sensing method

The working principle of the proposed catheter motion sensorallows for determining the catheter manipulation motions si-multaneously by using only one camera. The flow diagram forcatheter-sensing algorithm is summarized in Fig. 4. To use acamera to determine the catheter’s movements, an unavoid-able problem is the lens distortion resulted from the reasons ofmanufacturing. In practice, two main distortions, the radialdistortions generated due to the shape of lens and the tangen-tial distortions resulting from the assembly process of thecamera as a whole, should be considered (Kaehler andBradski 2008). Zhang’s method (Zhang 2002) is thus appliedto calibrate the camera in order to calculate both the camera’sgeometrical model and the distortion model of the lens. Thecalibration procedure starts with the collection of multipleimages of a planar pattern captured from a variety of anglesand positions. The correspondences between the points in the3D world coordinates and the 2D image points in the captured

multiple images are used to solve the camera parameters.Distortion parameters are employed to compute a distortionmap that is then used to correct the images. The coordinatevalues of the detected feature points are transformed fromtheir original coordinates to the corresponding undistorted co-ordinates based on the distortion map.

The next step is to detect the feature points of the passivemarker attached to the plastic hollow cylinder for computingthe translational and rotational motions of the input catheter.The corner points, which are detected as the candidate featurepoints including the required feature points in the passivemarker, are computed within the ROI (region of interest)based on the FAST (Features from Accelerated SegmentTest) feature detector developed by Rosten and Drummond(Rosten and Drummond 2005; Rosten et al. 2010). Actually,the movable range of the input catheter in the captured framesis limited to the middle section based on the structure of thecatheter holders. The mask area (shown in Fig. 3b) is thusintroduced to specify the ROI in which the corner points aredetected. The most promising advantage of the FAST cornerdetector is its computational efficiency. It is thus very suitableto detect the candidate points in real time due its high-speedperformance. Also, the reduced detection region considerablyshortens the computation time of the detection algorithm. TheFAST detector classifies whetheter a pixel point p is a corner

Fig. 4 The flow diagram of the proposed catheter-sensing methodFig. 3 The a distribution rule of the feature-point groups A, B, C and Dand the b marker attached to the hollow plastic cylinder

76 Page 4 of 11 Biomed Microdevices (2018) 20:76

by examining a circle of 16 pixels surrounding point p. Thepixels in the circle are clockwise labeled from 1 to 16 with theascending order. The number of the contiguous pixels in thecircle, which are all above than the intensity of the point p plusa thresholding value εd or all below the intensity of p minusthe thresholding value, are computated as shown in (1):

N ¼ ∑∀x∈SI xð Þ > I pð Þ þ εd∑∀x∈SI xð Þ < I pð Þ−εd

�ð1Þ

where p is a candidate corner point, I(p) presents the intensityof point p, S is a set of contiguous pixels in the surroundingcircle of point p, and εd presents the thresholding value. Thepixel point p is determined as the corner point if N is equal toor greater than 12. The computation speed can be optimizedby examining four example pixels (pixel 1, 9, 5 and 13) toreject non-corner points more quickly. At least three pixels inthe four example pixels should have brighter or darker inten-sity values than that of the candidate corner point becausethere should be at least 12 contiguous pixels which are allabove or below the candidate corner’s intensity.

The rectified frames are converted into grayscale images,and the FAST detector traverses the specified region to findthe candidate feature points. The original point of the imagecoordinate is defined at the upper left part of the capturedframes. The detected candidate feature points are first sortedby the Bubble Sort algorithm from small to large according tothe points’Y-coordinate values. The desired feature points arethen extracted from noise points by the distribution rules of theblack feature points in the passive marker that every two blackfeature points deployed in the same group have a 5-mm-spacing and four black points in the same group have almostequal coordinate values in the Y direction. Two groups ofblack feature points (e.g. group A&B or group A&D) willappear in an image at the same time according to the distribu-tion of the 4-feature-point groups while rotating the inputcatheter. Thus, two groups of the feature points are detectedfrom the consecutive images except in the critical case. Wegive priority to the group with smaller Y-coordinate value (i.e.active 4-feature-point group) to compute the axial and radialdisplacements (Δx and Δy) of the input catheter.

The sub-pixel coordinate values of the active 4-feature-point group of the passive marker are then calculated by in-terpolating the intensity surrounding the pixel coordinates ofthe four detected black feature points in order to achieve bettersensing accuracy. In our case, one pixel includes 10 sub-pixelsbased on the linear interpolation algorithm. To determine theaxial motion of the input catheter, the sub-pixel displacementvariations of the four detected feature points in the X directionare computed. Assuming that the active 4-feature-point groupis located at P1, P2, P3 and P4 in the current frame and appears

onP01, P

02, P

03 and P

04 in the next frame (shown in Fig. 5a), the

average value of the displacement variations determines the

axial motion of the input catheter as (2) shows. Similarly, theaverage displacement variation in Y direction is computed tomeasure the radial movement as (3) shows:

Δxpixel ¼ Δx1 þ Δx2 þ Δx3 þ Δx4ð Þ=4 ð2Þ

Δypixel ¼ Δy1 þ Δy2 þ Δy3 þ Δy4ð Þ=4 ð3Þ

where Δxpixel and Δypixel present the average sub-pixel dis-placement variations of four black feature points in the Xand Y directions respectively.

Next is to build the relationship between the pixel unit andthe millimeter unit. During the calibration process, a motor-ized linear module was used to drive the catheter to a presetdistance. The optical encoder recorded the catheter’s displace-ment in millimeter (dmm) whilst its corresponding displace-ment in the image coordinate system (dpixel) was calculatedbased on (2). We repeat the calibration process 10 times andthe average value is determined as the mm/pixel ratio α. Theparameter α defines the relationship between millimeters andpixels. It is independent of the pushing or pulling speeds. It isthe result obtained by dividing the translational displacementin millimeter unit by its corresponding displacement in pixelunit. The parameter α is then determined by (4):

α ¼ dmmdpixel

ð4Þ

Thus, the translational displacement in millimeter unit(Δxmm) can be directly calculated according to Δxpixel by (5).As for the radial movements, we assume that the rotational

Fig. 5 The a axial and radial motions of the input catheter and the bcomputation for the rotational angles

Biomed Microdevices (2018) 20:76 Page 5 of 11 76

angles of the input catheter generated by anticlockwise twistare positive. When the catheter is twisted in an anticlockwisedirection, the active feature-point group will move from top tobottom in the captured images. The radial motion (Δθ°) of theinput catheter can be computed by (6) based on the geometri-cal relationship shown in Fig. 5b.

Δxmm ¼ Δxpixel∙α ð5Þ

Δθ∘ ¼ arcsinm−y0

r⋅a−arcsin

m−yr⋅a

m ¼ Ctl þ Cbl þ Ctr þ Cbr

4

ð6Þ

where r is the radius of the plastic hollow cylinder, y and y′

are the pixel coordinate values of the active feature-pointgroup in previous frame and in the current frame respec-tively, and m is the computed Y coordinate value of thecenter line of the plastic cylinder. m is initialized by deter-mining the Y coordinates (ctl, cbl, ctr, and cbr) of the fourcorners of the plastic cylinder in the first frame.Geometrical prior knowledge can result in fast detectionfor the four corners once that the active feature-point groupis determined (shown in Fig. 7c).

During the measurement procedure, two groups of featurepoints are simultaneously detected in the captured images(shown in the Phase A of Fig. 6) while only one feature-point group is sensed in the critical situation (shown in thePhase B of Fig. 6). In the case of Phase A, the feature-pointgroup with smaller coordinate values in the Y direction is usedto measure the translational and rotational movements prior tothe other group. The measurement range of the rotational an-gles for each group is thus about 0° ~ 90°. To achieve betterperformance in sensing rotational angles, the compensationmethod is triggered when one feature-point group reaches itstravel limit and the other group starts to measure the radialmotion (i.e. in the critical situation). Assuming that the groupA is going to reach its top travel limit in the current frame anddisappear in the next frame, and the group B will be the activefeature-point group and be detected in the next frame, the

compensation method will be triggered by large variation ofY-coordinate values (the Y-coordinate value of group B ismuch larger than that of group A). The proposed catheter-sensing method is capable of recording the coordinate varia-tion values of group A in the last three frames. Thus, theaverage value of the three variation values will be compensat-ed to the rotational measurement results. The compensationmethod is an effective way to improve the measurement ac-curacy of the radial motion because the rotational operationsare persistent and do not occur fast.

3 Experimental results

The camera is connected to the computer console via a USBport. The image processing algorithms are implemented byusing OpenCV, which is an open-source library of program-ming functions mainly aimed at real-time computer vision. Inorder to evaluate the performance of the proposed catheter-sensing method, three experiments were designed andconducted.

3.1 The camera calibration

The camera was calibrated based on a 6 * 9 chessboard. Byviewing this flat chessboard from a variety of distances andangles (20 times in our case), the intrinsic matrix and distor-tion vector were calculated. In practice, we employed threeradial distortion terms (k1, k2, k3) and two tangential distortionparameters (p1, p2) to characterize a distortion map for recti-fying the frames captured by the camera, as (7) shows:

x0 ¼ x 1þ k1r2 þ k2r4 þ k3r6

� �þ 2p1xyþp2 r2 þ 2x2

� �y0 ¼ y 1þ k1r2 þ k2r4 þ k3r6

� �þ 2p2xyþp1 r2 þ 2y2

� �ð7Þ

where r2 = x2 + y2 and (x, y) is the coordinate of the pixels onthe original images, and (x′, y′) is the new coordinate value as aresult of the rectification.

The distortion vector and the intrinsic parameters, resultingfrom Zhang’s algorithm for the camera, are listed in Table 1. fxand fy (pixel units) are the focal lengths of the camera in X andY directions respectively. cx and cy are two parameters used to

Fig. 6 The critical situation for the rotational motion measurement

Table 1 Distortion and intrinsic parameters for the camera

Distortion vector k1 k2 p1 p2 k3−0.147 0.869 0 0 0

Intrinsic matrix fx fy α cx cy1103.84 1103.84 0 319.5 239.5

76 Page 6 of 11 Biomed Microdevices (2018) 20:76

model the displacement coordinates with respect to the opticaxis and the center of the projection screen.

3.2 Evaluation of feature points detection algorithm

According to the feature detection algorithm described above,the results of the proposed catheter-sensing method are shownin Figs. 7 and 8. We first detected the best corner points basedon the FAST algorithm in the mask region (shown in Fig. 7a).The noise points were then removed based on the distributionrules of the four black feature points (shown in Fig. 7b). Theaverage displacement variation of the four feature points ofgroup A, B, C or D in the X direction was used to measure theaxial motion of the catheter.

For the radial motion, the geometrical model determinedrotational angles based on the average displacement of thefour feature points in the Y direction. Group A was firstused to measure the rotational angles of the input catheter(Fig. 8a). When the group A reached its travel limit anddisappeared in the image, group B was then detected as theactive feature-point group to measure the radial motion ofthe input catheter instead of group A (Fig. 8b and c). Thecompensation computation was triggered in the critical sit-uation when the active feature-point group was changedfrom group A to group B. Similarly, group C continuedto sense the rotational angles of the input catheter whengroup B reached its travel limit (Fig. 8d). The algorithmconsistently selected the group with much smaller Y-coordinate value as the active feature-point group.

Additionally, the feature-point detection experiments wereconducted in different lighting conditions. The experimentalresults showed that the proposed detection algorithm was sta-ble enough and immune to the lighting intensity variation.

3.3 Evaluation of the catheter-sensing method

To evaluate the measurement performance, we employed astepper motor with an encoder (VEXTA ASM46AA, 1000count-per-revolution quadrature encoder) to pull, push andtwist the catheter, and compared the data obtained by the en-coder with the translational and rotational results computed bythe proposed camera based catheter-sensing method. The ex-periment setup is shown in Fig. 9a and b. The encoder datawas regarded as the benchmark. The pushing and pulling ex-periments were first conducted at an approximately constantvelocity of 8.4 mm/s. The comparative results are plotted as afunction of time (shown in Fig. 10a and b).

From the experimental results, 362 frames were cap-tured by the camera and only 1 frame was failed to detectthe feature points. The error frame would not damage themeasurement results because only the variation of dis-placement contributed to the detected displacement. Theabsolute error of ±0.23 mm and the relative error of

±1.8% were measured, and the mean absolute error(MAE) was 0.15 mm when repeating this experiment fivetimes over 500 mm.

Fig. 7 The experimental results of the feature detection algorithm: a thedetection of the 80 strongest corner points; b the detection of the activefeature-point group; c the detection for the corners of the plastic cylinder

Biomed Microdevices (2018) 20:76 Page 7 of 11 76

Additionally, the experiments with the velocity of16.8 mm/s, 25.2 mm/s, 33.6 mm/s and 42 mm/s were carriedout. Similarly, we repeated the pulling and pushing experi-ments five times for each velocity. During the experimentswith the velocity of 42mm/s, 920 frameswere totally capturedby the camera with 100% success rate for the feature points’detection. The zoomed-in data of a round trip is shown inFig. 10c and d. The measurement results of the experimentswith different velocities were summarized in Table 2.

For the performance evaluation of the radial motion, theinput catheter was twisted by the stepper motor at the angularvelocities of 95.5°/s and 117.5°/s respectively. We alsoregarded the rotational angles obtained from the encoder asthe benchmark. The rotational experiments were repeated five

Fig. 9 The experimental setup for a pulling and pushing experiments andb twisting experiments

�Fig. 8 The experimental results of the rotational measurement: a thedetection of the active feature-point group A; b the top position of thefeature-point group A; c the detection of the feature-point group B; d thetop position of the feature-point group B

76 Page 8 of 11 Biomed Microdevices (2018) 20:76

times for each angular velocity over 3600°. The 2240 framesand 1720 frames were totally captured by the camera for twoexperiments with success rates of 99.6 and 99.4% respective-ly. For the rotational experiments with the angular velocity of95.5°/s, the absolute error of ±2.16° and the relative absoluteerror of ±3.20% were measured, and the mean absolute errorwas 1.98°. As for the angular velocity of 117.5°/s, the absoluteerror of ±2.15° and the relative absolute error of ±3.42% werereported with the mean absolute error of 2.23°. The zoomed-incomparative results of the first 360° were shown in Fig. 11.

4 Discussion

Compared to the existing measurement methods applied incommercial products and research achievements for recordingthe experienced surgeons’ catheter manipulation trajectories,the proposed catheter-sensing method has many potential ad-vantages on measuring the catheter motions. Firstly, the de-signed passive marker can be easily detected to allow simul-taneously sensing the axial and radial motions of the inputcatheter with low computation cost. The camera based sensingprinciple can realize contactless measurement rather than thetwo-optical-encoder-based or electromagnetic-sensor-basedmethods which could contaminate the operators’ tactile feed-back in the bedside catheterization procedures. Secondly, theoperators are allowed to directly manipulate a real catheter,instead of the Bmetal rod^, by applying conventional pulling,pushing and twisting operations used in actual intravascularinterventions, unlike the commercial products employing 3Djoystick, navigation buttons, the PHANTOM Omni, orNintendo Wii to manipulate the catheters, which are not in amanner that is in accordance with conventional catheter ma-nipulation techniques. Thirdly, the proposed sensing methodis immune to the size of the input catheter. The sensing prin-ciple was validated based on a 6F catheter in this paper. It canbe easily extended to the catheters with larger or smaller sizes

(a)

(b)

(c)

(d)

0

5

10

15

20

25

30

35

40

45

50

55

0 1 2 3 4 5 6

Disp

lace

men

t(m

m)

Time (s)

EncoderCamera

-55-50-45-40-35-30-25-20-15-10-50

0 1 2 3 4 5 6

Disp

lace

men

t(m

m)

Time (s)

EncoderCamera

05

1015202530354045505560

0 0.25 0.5 0.75 1 1.25 1.5

Disp

lace

men

t(m

m)

Time (s)

EncoderCamera

-60-55-50-45-40-35-30-25-20-15-10-50

0 0.25 0.5 0.75 1 1.25 1.5

Disp

lace

men

t(m

m)

Time (s)

EncoderCamera

Detec�on failed frame

Table 2 Summary of the performance in the pushing and pullingexperiments

Velocity (mm/s) 16.8 25.2 33.6 42.0

Absolute error (mm) ±0.19 ±0.24 ±0.30 ±0.25

Relative error (%) ±2.1 ±2.8 ±3.4 ±2.3

MAE (mm) 0.11 0.21 0.27 0.22

Total frames 1920 1360 1080 920

Success rate (%) 100 99.26 99.73 100

�Fig. 10 The comparative experimental results: a insertion at a velocity of8.4 mm/s; b extraction at a velocity of 8.4 mm/s; c insertion at the velocityof 42 mm/s; d extraction at the velocity of 42 mm/s

Biomed Microdevices (2018) 20:76 Page 9 of 11 76

because we only need to guarantee that the new input cathetercan pass through and be fixed to the over-tube. The sensingmethod is thus applicable not only to catheterization proce-dures but also to similar interventions (e.g. endoscopic proce-dure). In addition, the measurement errors resulting from thesmooth surface of the input catheter can be prevented. Theproposed passive marker is used to sense the catheter motionsrather than the feature points on the catheter surface. Finally,the proposed catheter sensing method is more cost-effectivethan the other existing measurement methods.

The limitation of the proposed catheter-sensing method isthe translational stroke length. The proposed sensing principlecan allow no limitation to the rotational measurement, butprovide approximately 150 mm in the axial motion of theinput catheter. The catheter has to be repositioned when itreaches its travel limit. Even though the current axial stokeleng th can reach the requ i rement accord ing to(Srimathveeravalli et al. 2010), the amount of the axial motionshould only be limited by the length of the input catheter usedin the measurement procedures. To expand the translationalstroke length, the distance between two catheter holders

should be firstly enlarged, and several markers with plastichollow cylinders (only one in current practice) are required.When one marker reaches its travel limit and disappears in theimage view, another marker should be detected, and continueto measure the axial and radial motions of the input catheter.

5 Conclusion and future work

In this paper, a new catheter-sensing method was proposed todetermine the catheter manipulation skills of experienced sur-geons whilst still allowing experts to use the natural catheter-ization skills based on the analysis of requirements and thestate of the art methods for measuring the catheter motions. Apassive marker with four feature-point groups was designedand attached to a hollow cylinder with a much larger radius. Acamera was employed to realize the contactless measurementfor the axial and radial motions of the input catheter by track-ing the feature points of the passive marker. A series of exper-iments were carried out to evaluate the performance of theproposed catheter-sensing method.

In our future work, a database and an assessment systemwill be built based on the proposed catheter-sensingmethod inorder to assess surgeons’ operation gestures, catheter inser-tion, extraction and rotation speeds, and the operation frequen-cy of surgeons’ hand.

Acknowledgments This research is partly supported by National NaturalScience Foundation of China (61375094), National High Tech. Researchand Development Program of China (No.2015AA043202), and SPSKAKENHI Grant Number 15 K2120.

References

R. Aggarwal, S.A. Black, J.R. Hance, A. Darzi, N.J.W. Cheshire, Virtualreality simulation training can improve inexperienced surgeons’endovascular skills. Eur. J. Vasc. Endovasc. Surg. 31(6), 588–593(2006)

L. J. Brattain, C. Floryan, O. P. Hauser, M. Nguyen, R. J. Yong, S. B.Kesner, S. B. Corn, C. J. Walsh, Simple and effective ultrasoundneedle guidance system, in Annual International Conference of theIEEE Engineering in Medicine and Biology Society, Boston, Aug./Sep. 2011, pp. 8090–8093

G. Chen, C. Lin, C. Li, K. Hsieh, Y. Du, T. Chen, Virtual-reality simulatorsystem for double interventional cardiac catheterization using hapticforce producer with visual feedback. Comput. Electr. Eng. (2015).https://doi.org/10.1016/j.compeleceng.2015.11.013

Z. Feng, G. Bian, X. Xie, Z. Hou, J. Hao, Design and evaluation of a bio-inspired robotic hand for percutaneous coronary intervention, inProceedings of 2015 IEEE International Conference on Roboticsand Automation, Seattle, Washington, May 26–30 2015, pp.5338–5343

J. Guo, S. Guo, N. Xiao, X. Ma, S. Yoshida, T. Tamiya, M. Kawanichi, Anovel robotic catheter system with force and visual feedback forvascular interventional surgery. Int. J. of Mechatronics andAutomation 2(1), 15–24 (2012)

(a)

(b)

0306090

120150180210240270300330360390

0 0.5 1 1.5 2 2.5 3 3.5

Rota

�ona

l Ang

les (

degr

ee)

Time (s)

Encoder

Camera

-390-360-330-300-270-240-210-180-150-120-90-60-300

0 0.5 1 1.5 2 2.5 3 3.5

Rota

�ona

l Ang

les (

degr

ee)

Time (s)

EncoderCamera

Fig. 11 The comparative results of the rotational motion at the angularvelocity of 95.5°/s: a 360-degree rotation in clockwise direction; b 360-degree rotation in anticlockwise direction

76 Page 10 of 11 Biomed Microdevices (2018) 20:76

J. Guo, S. Guo, T. Dauteuille, AVR-based Training System for VascularInterventional Surgery, in Proceedings of 2013 ICME InternationalConference on Complex Medical Engineering (ICME CME 2013),Beijing, China, May 25–28, 2013, pp. 575–579

J. Guo, S. Guo, T. Tamiya, H. Hirata, H. Ishihara, Design and per-formance evaluation of a master controller for endovascularcatheterization. Int. J. Comput. Assist. Radiol. Surg. 11(1),119–131 (2016)

S. Ikeda, F. Arai, T. Fukuda, M. Negoro, K. Irie, I. Takahashi, Patient-Specific Neurovascular Simulator for Evaluating the Performance ofMedical Robots and Instruments, in Proceedings of the 2006 IEEEInternational Conference on Robotics and Automation, Orlando,Florida, 2006, pp. 625–630

A. Kaehler, G. Bradski, Learning OpenCV. (Publisher: O'Reilly Media,2008)

H. Kodama, C. Tercero, K. Ooe, C. Shi, S. Ikeda, T. Fukuda, F. Arai, M.Negoro, I. Takahashi, G. Kwon, 2-D optical encoding of cathetermotion and cyber-physical system for technical skills measurementand quantitative evaluation in endovascular surgery, in Proceedingsof 2012 IEEE/RSJ International Conference on Intelligent Robotsand Systems, Vilamoura, Oct. 7–12, 2012, pp. 3565–3570

H. Kodama, C. Shi, M. Kojima, S. Ikeda, F. Arai, I. Takahashi, M.Negoro, T. Fukuda, Catheter manipulation training system basedon quantitative measurement of catheter insertion and rotation.Adv. Robot. 28(19), 1321–1328 (2014)

S. Li, Q. Wang, Q. Meng, Y. Chui, J. Qin, P. Heng, A catheterization-training simulator based on a fast multigrid solver. IEEE Comput.Graph. Appl. 32(6), 56–70 (2012)

W. Lu, W. Xu, J. Zhang, D. Liu, D. Wang, P. Jia, Z. Li, T. Wang, D.Zhang, Z. Tian, Y. Zeng, Application study of medical robots invascular intervention. Int. J. Med. Rob. Comput. Assisted Surg.7(3), 361–366 (2011)

X. Ma, S. Guo, N. Xiao, S. Yoshida, T. Tamiya, Evaluating performanceof a novel developed robotic catheter manipulating system. J.Microbio. Robot. 8(3–4), 133–143 (2013)

M. Negoro, M. Tanimoto, F. Arai, T. Fukuda, K. Fukasaku, I. Takahashi,S. Miyachi, An intelligent catheter system robotic controlled cathe-ter system. Interv. Neuroradiol. 7, 111–113 (2002)

H. Rafii-Tari, C.J. Payne, G. Yang, Current and emerging robot-assistedendovascular catheterization technologies: A review. Ann. Biomed.Eng. 42(4), 697–715 (2014)

C.V. Riga, C.D. Bicknell, A. Rolls, N.J. Cheshire, M.S. Hamady,Robot-assisted fenestrated endovascular aneurysm repair(FEVAR) using the Magellan system. J. Vasc. Interv. Radiol.24(2), 191–196 (2013)

E. Rosten, T. Drummond, Fusing points and lines for high performancetracking, in Tenth IEEE International Conference on ComputerVision, Beijing, China, Oct. 17–21, 2005, pp. 1508–1515

E. Rosten, R. Porter, T. Drummond, Faster and better: Amachine learningapproach to corner detection. IEEE Trans. Pattern Anal. Mach.Intell. 32(1), 105–119 (2010)

W. Saliba, V.Y. Reddy, O. Wazni, J.E. Cummings, J.D. Burkhardt, M.Haissaguerre, J. Kautzner, P. Peichl, P. Neuzil, V. Schibgilla, G.Noelker, J. Brachmann, L.D. Biase, C. Barrett, P. Jais, A. Natale,Atrial fibrillation ablation using a robotic catheter remote controlsystem: Initial human experience and long-term follow-up results.J. Am. Coll. Cardiol. 51(25), 2407–2411 (2008)

E. Samur, L. Flaction, H. Bleuler, Design and evaluation of a novel hapticinterface for endoscopic simulation. IEEE Trans. Haptics 5(4),(2012)

C. Shi, C. Tercero, S. Ikeda, K. Ooe, T. Fukuda, K. Komori, K.Yamamoto, In vitro three-dimensional aortic vasculature modelingbased on sensor fusion between intravascular ultrasound and mag-netic tracker. Int. J. Med. Rob. Comput. Assisted Surg. 8(3), 291–299 (2012)

G. Srimathveeravalli, T. Kesavadas, X. Li, Design and fabrication of arobotic mechanism for remote steering and positioning of interven-tional devices. Int. J. Med. Rob. Comput. Assisted Surg. 6, 160–170(2010)

J. Stoll, P. E. Dupont, Passive markers for ultrasound tracking of surgicalinstruments, in Medical Image Computing and Computer-AssistedIntervention (MICCAI), Springer, 2005, pp.41–48

M. Tanimoto, F. Arai, T. Fukuda, H. Iwata, K. Itoigawa, Y. Gotoh, M.Hashimoto, M. Negoro, Micro force sensor for intravascular neuro-surgery, in Proceeding of the IEEE International Conference onRobotics and Automation, April 20–25, Albuquerque, NM, 1997,pp. 1561–1566

C. Tercero, S. Ikeda, T. Uchiyama, T. Fukuda, F. Arai, Y. Okada, Y. Ono,R. Hattori, T. Yamamoto, M. Negoro, I. Takahashi, Autonomouscatheter insertion system using magnetic motion capture sensor forendovascular surgery. Int. J. Med. Rob. Comput. Assisted Surg.3(1), 52–58 (2007)

C. Tercero, F. Arai, M. Negoro, I. Takahashi, Numerical comparison ofcatheter insertion trajectory within blood vessel model using imageprocessing, in 2010 International Symposium on MicroNanoMechatronics and Human Science, Nagoya, Japan, Nov.2010, pp. 378–383

C. Tercero, H. Kodama, C. Shi, K. Ooe, S. Ikeda, T. Fukuda, F. Arai, M.Negoro, G. Kwon, Z. Najdovski, Technical skills measurementbased on a cyber-physical system for endovascular surgery simula-tion. Int. J. Med. Rob. Comput. Assisted Surg. 9(3), e25–e33 (2013)

Y. Thakui, J. H. Cakiroglu, D.W. Holdsworth,M. Drangova, A device forreal-time measurement of catheter-motion and input to a catheternavigation system, in Medical Imaging 2007: Visualization andImage-Guided Procedures, Vol. 6509, SPIE Proceedings, SanDiego, CA, USA, Feb. 2007

Y. Thakui, J.S. Bax, D.W. Holdsworth, M. Drangova, Design and perfor-mance evaluation of a remote catheter navigation system. IEEETrans. Biomed. Eng. 56(7), 1901–1908 (2009)

J. Wang, T. Ohya, H. Liao, I. Sakuma, T. Wang, I. Tohnai, T. Iwai,Intravascular catheter navigation using path planning and virtualvisual feedback for oral cancer treatment. Int. J. Med. Rob.Comput. Assisted Surg. 7, 214–224 (2011)

D. Wang, C. Yang, Y. Zhang, J. Xiao, Toward in-vivo force and motionmeasurement for vascular surgery. IEEE Trans. Instrum. Meas.63(8), 1975–1982 (2014)

D. Wei, S. Hasegawa, K. Takahashi, E. Ryzhii, X. Zhu, A virtual realityfor catheter-based EPS based on whole-heart model. InternationalJournal of Bioelectromagnetism 11(1), 2–6 (2009)

Y. Yan, D. Chen, H. Yin, Optics based motion measurement for a catheternavigation system: A novel and low cost approach. InternationalConference on Intelligent Robotics and Applications, Shanghai,China, 10–12 November 2010

Z. Zhang, A flexible new technique for camera calibration. IEEE Trans.Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2002)

Biomed Microdevices (2018) 20:76 Page 11 of 11 76