Multipoint Collision Detection Algorithm for Bone Milling and

Multipoint Collision Detection Algorithm for Bone Milling and Drilling

in a Haptic VR Surgery Simulator

N I C O L A S A S E N J O O S O R I O

Master of Science Thesis Stockholm, Sweden 2008

Multipoint Collision Detection Algorithm for Bone Milling and Drilling

in a Haptic VR Surgery Simulator

N I C O L A S A S E N J O O S O R I O

Master’s Thesis in Computer Science (30 ECTS credits) at the School of Media Technology Royal Institute of Technology year 2008 Supervisor at CSC was Lars Kjelldahl Examiner was Lars Kjelldahl TRITA-CSC-E 2008:068 ISRN-KTH/CSC/E--08/068--SE ISSN-1653-5715 Royal Institute of Technology School of Computer Science and Communication KTH CSC SE-100 44 Stockholm, Sweden URL: www.csc.kth.se

Multipoint collision detection algorithm for bone milling and drilling in a haptic VR

surgery simulator

ABSTRACT This report describes how a multipoint collision detection algorithm -for a haptic surgery simulator- is developed, implemented and validated. The thesis work is part of a project initiated by the Center for Technology and Health Care (CTV), which is a collaboration between the Royal Institute of Technology (KTH) and the Karolinska Institute (KI), Stockholm, Sweden. The aim is to enhance the haptic part of the existing surgery simulator. Two different algorithms where developed, compared, tested and analyzed. The result of both algorithms was a perceptible improvement of the haptic feedback, however each algorithm has its advantages and disadvantages.

En algoritm som känner av multipla kollisionspunkter för benfräsning och

benborrning i en haptisk kirurgsimulator

SAMMANFATTNING Denna rapport beskriver hur en algoritm, som känner av multipla kollisionspunkter för en haptisk kirurgsimulator, är utvecklad, implementerad och validerad. Examensarbetet är en del av ett projekt som drivs av Centrum för Teknik i Vården (CTV), som i sin tur ett samarbete mellan Kungliga Tekniska Högskolan (KTH) och Karolinska Institutet (KI), Stockholm, Sverige. Målet är att förbättra den haptiska delen av den existerande kirurgsimulatorn. Man utvecklade två olika algoritmer som jämfördes, testades och analyserades. Resultatet av de båda algoritmerna var en förnimbar förbättring av den haptiska återkopplingen, dock har varje algoritm sina fördelar och nackdelar.

Un algoritmo que detecta colisión de multipuntos para fresar y taladrar huesos en

un entorno háptico de realidad virtual en un simulador quirúrgico.

RESUMEN Este informe describe como un algoritmo que detecta colisión de multipuntos, para un simulador háptico de cirugía que es desarrollado, implementado y convalidado. La tesis forma parte de un proyecto desarrollado por el Centro de Tecnología para la Salud (CTV), que es una colaboración entre el Instituto Real de Tecnología (KTH) y el Instituto Carolingio (KI), Estocolmo, Suecia. El objetivo es mejorar la parte háptica del actual simulador quirúrgico. Se desarrollaron dos algoritmos diferentes que fueron comparados, probados y analizados. El resultado de los dos algoritmos fue un mejoramiento perceptible de la retroalimentación háptica, sin embargo cada algoritmo tiene sus ventajas y desventajas.

KEYWORDS:

Haptic feedback, collision detection, surgical simulator, medical simulator, virtual reality, 3D visualization, algorithms, forces.

ACKNOWLEDGEMENTS

I would like to thank my beloved parents for supporting me and guiding me through

the path of the every day life. My only brother for just being there and letting me

know that I am not alone in this world.

Secondly, I would like to thank my supervisor PhD. candidate Magnus G. Eriksson

at the Mechatronics Lab at the Royal Institute of Technology (KTH) for all the help

during this thesis and making me belive that this work is meaningful.

Finally, I would like to thank my family: my girlfriend Sol, my eldest daughter Abril

and my new baby girl Luz Melina for giving me the strength to finish this thesis and

for not giving up on me during difficult times.

Nicolás Asenjo Osorio, Stockholm, Sweden, May 2008

LIST OF ACRONYMS KTH The Royal Institute of Technology KI Karolinska Institute PhD Doctor of Philosophy MPC Multi Point Collision FF Force Feedback DOF Degrees of Freedom XML eXtensible Markup Language X3D The ISO standards for real-time 3D computer graphics,

XML-syntax CT Computer Tomography MRI Magnetic Resonance Imaging DICOM Digital Imaging and Communication in Medicine

TABLE OF CONTENTS

1 INTRODUCTION ______________________________________________1

1.1 Background__________________________________________________________________ 1

1.2 Haptics______________________________________________________________________ 3

1.3 Definitions ___________________________________________________________________ 4

1.4 Purpose and goals_____________________________________________________________ 5

1.5 Problem Description __________________________________________________________ 6

1.6 Equipment___________________________________________________________________ 7

1.7 Delimitations_________________________________________________________________ 8

1.8 Target Audience ______________________________________________________________ 9

1.9 Work Preformed _____________________________________________________________ 9

1.10 Overview ___________________________________________________________________ 10

2 RELATED WORK ____________________________________________11

2.1 The virtual and haptic milling surgery simulator __________________________________ 12 2.1.1 Using patient-specific DICOM data ____________________________________________ 13 2.1.2 Graphic rendering __________________________________________________________ 14 2.1.3 Haptic rendering ___________________________________________________________ 15

3 THE ALGORITHMS __________________________________________17

3.1 The Line Approximation algorithm _____________________________________________ 17 3.1.1 Discretization of the tool _____________________________________________________ 17 3.1.2 Collision Detection _________________________________________________________ 18 3.1.3 Find Intersection Point ______________________________________________________ 19

3.1.3.1 If no collision at time step t-1 _____________________________________________ 20 3.1.3.1.1 Ascertain Active Vertices by Translation/Rotation method____________________ 23 3.1.3.1.2 Ascertain Active Vertices by Ray Casting method __________________________ 25

3.1.3.2 If collision at time step t-1 _______________________________________________ 27 3.1.4 Calculate the haptic Force- and Torque feedback __________________________________ 28 3.1.5 Placement of the Proxy for correct visualization___________________________________ 31

3.2 The Surface Approximation Algorithm __________________________________________ 33 3.2.1 Discretization of the tool _____________________________________________________ 34 3.2.2 Collision Detection _________________________________________________________ 35 3.2.3 Calculate the haptic Force- and Torque feedback __________________________________ 36

4 IMPLEMENTATION __________________________________________41

4.1 Workstation ________________________________________________________________ 41 4.1.1 Integration & implementation _________________________________________________ 41 4.1.2 Testing & verifying _________________________________________________________ 42

4.2 Software ___________________________________________________________________ 43

5 RESULTS __________________________________________________45

5.1 Verification and analysis ______________________________________________________ 45 5.1.1 Evaluation of the Line Approximation Algorithm _________________________________ 47 5.1.2 Evaluation of the Surface Approximation Algorithm _______________________________ 50

5.2 Test Application _____________________________________________________________ 52 5.2.1 Test results for the Line approximation algorithm _________________________________ 54 5.2.2 Test results for the Surface approximation algorithm _______________________________ 56

6 CONCLUSION_______________________________________________59

6.1 Discussion __________________________________________________________________ 59

REFERENCES __________________________________________________61

APPENDIX______________________________________________________63

Overview of “The Line Algorithm” _____________________________________________________ 63

Overview of “The Surface Algorithm” __________________________________________________ 64

Chapter 1 Introduction ________________________________________________________________

1

1 Introduction

This thesis is a further development of a haptic and virtual reality simulator. The

simulator is a prototype for simulating the bone milling and material removal process

occurring in several operations, such as temporal bone surgery or dental milling. This

project is realized for the Mechatronics Lab at the Royal Institute of Technology (KTH)

and is funded by the Center for Technology and Health Care (CTV), which is cooperation

between KTH and the Karolinska Institute (KI), Stockholm, Sweden.

Magnus G. Eriksson who is a PhD candidate is conducting this project at the

Mechatronics Lab at the Department of Machine Design at KTH. Eriksson’s supervisor is

Professor Jan Wikander.

1.1 Background

For removal of malignant tumors in specific parts of the human head, the surgery consists

of opening a hole in the skull bone, and in some cases making a path along the inside of

the bone. Nowadays, the surgeon drills a path very carefully with a small hand held mill.

When this is done the tumor can be reached without affecting the brain more than

necessary. Commonly this path is usually located in parts where the skull bone is

geometrically complicated. That is surrounded by neurons, brain tissue and critical parts

of the human nervous system. This makes the drilling phase of this type of operation very

complicated and time consuming. Reduction of operation time by only a few percent

would in the long run save resources. Also one can consider the invaluable knowledge

regarding potential complications, risks and other vital factors gained prior to a real

operation [8].


2

Today computers have become an invaluable tool for visualization of data for different

type of simulations and calculations, everything from simple tables and diagrams, to

complex high resolution graphics rendering of computer games. Nowadays when the

computer performance and capacity increases day by day and successful simulators like

for the aviation among others, the computer simulations and visualizations has reached

and entered a new phase.

The time has come for implementation of haptics, the sense of touch, into 3D worlds and

virtual realities (VR). Unlike the simulators of the aeronautics industry, where the main

objective is to avoid collision with objects, the haptic environment does the opposite; to

collide with the virtual objects and to make it “feel” real is essential. These are fields that

are still relatively unexplored and have only been around approximately for 10-15 years.

One of the areas of interest is the medical field. The risk of training on real patients and

the changeover to endoscopic procedures have contributed to the interest of development

of haptics and VR simulators.

Not only is the interest for the medical field but also for several other areas, e.g. 3D

computer games, to make computers accessible for the blind and visually impaired,

remote sexual relations (“teledildonics”) [19, 20], robotics and arts, among others.

The main reasons for further development of this haptic and VR simulator are [1]:

1. More endoscopic procedures in the surgical areas, makes the purpose of a haptic

VR environment very natural, because of the similarity of monitor-based

navigation and using similar equipment.

2. Simulator creates new training opportunities for surgical procedures that are

otherwise impossible to realize and can e.g. measure the operating skills of a

surgeon by using a computer-based tracking system to evaluate specific surgical

performance.

3. Using the simulator for planning actual surgeries will reduce errors and make the

surgeon feel more secure when performing a real operation.


3

4. It will be possible to practice and evaluate both new and ordinary operating

methods, which is impossible today due to the safety of the patient.

5. Instead of training on animal cadavers or other live or dead objects, the training

room can be moved from the operating room to the VR simulator. This can save a

considerable amount of expenses in the long run, for any hospital or organization.

6. A haptic VR environment can be the introduction of robot-assisted surgery. By

using robot-assisted surgery one can increase precision and safety of operations

and decrease the operating time.

The haptic field has yet to be explored and with creative minds find smart usage for it, the

future for haptics can come to be a meaningful every day thing. Even though most of the

above mention reasons haptics as a research tool is relatively young. We are far from

being revolutionized by haptic products in our every day life use, mainly because of the

expensive hardware; especially the sophisticated ones, constraining the smaller researcher

organizations and making it more difficult to come across meaningful investigation

funds. Imagining a future scenario using a haptic interface can be many. One example is

using a haptic machine or device when performing a remote surgical procedure. E.g.

where a surgeon can be at a remote address or point from the patient and still perform a

surgery. By integrating the Internet and sending haptic data information from peer to

peer. Unfortunately this is only real in the sci-fi movies or animations films.

1.2 Haptics

The word haptic comes from the Greek words haptesthai [14], haphe [4] or haptikos [9],

the sense of touch. The term haptics in today’s researching world is in its broadest sense

related to the study of touch and the cutaneous senses [9]. Research within the augmented

reality field has mainly focused on visual or acoustic procedures. Never the less, humans

can perceive information not only trough vision and sound, but also through haptics.


4

Researchers are focusing more and more on making use of the haptics in augmented

reality systems, while the haptic sensations are very intuitive [10].

1.3 Definitions

When talking about haptic feedback it can be broadly divided into two modalities:

vibrotactile and kinesthetic. Vibrotactile feedback stimulates the human subcutaneous

tissue. This is very common and used nowadays in mobile phones, video console game

pads, and certain touch panels. Kinesthetic feedback focuses on the gross movement of

the human body. It is being employed in programmable haptic knobs [7] or stylus [6],

video game steering wheels, medical simulation trainers and in virtual reality systems.

Force feedback (FF) is a term used to describe kinesthetic and/or vibrotactile feedback

[11]. In this report I will use the word “feedback” but usually referring to “force

feedback”.

Also the word “simulation” has a broad meaning. Depending on the context the word

simulation has a broad variety of meaning, e.g. simulation as in pretending or show, or a

terrorist “high jacking” simulation of an airplane, etc [12]. In this thesis the word

“simulation” will be used as in a computer science or medical simulators context. When

the simulator is executed, a virtual tooth and a pen like instrument (the stylus) appear in

our 3D world, which is on the screen, see Figure 1.

Figure 1. Screen shot of the simulator.


5

A proxy-probe-based method [1] is used in haptic algorithms. The probe is the real

position in the 3D space of the haptic device while the proxy is the virtual position of the

device, the representation of the tool on the screen. When the probe enters the surface of

a virtual object the proxy remains on the surface of the manipulated object. A force is

sent back to the haptic device, the probe, based on the distance between the probe and the

proxy.

The technique called ray-casting is used in this thesis. Ray-casting is in general a

technique that transforms a limited form of data into a 3D projection by tracing rays from

the view point into the viewing volume [13]. The main idea of using the ray-casting

technique, in this thesis, is to follow the projection of a vector from point A to point B.

The important thing is to understand that the principle of ray-casting is used.

When working with haptics, words as translation and orientation are frequently

encountered. This thesis is not an exception. Translation is related with the positioning (x,

y, and z) of objects or vertices in 3D space. Orientation is related with the rotation (yaw,

pitch, and roll) of objects or vertices in 3D space. As the movement along each of the

three axes (x, y, and z) is independent of each other and independent of the rotation about

any of these axes, then the motion has six degrees of freedom (6-DOF) [15].

1.4 Purpose and goals

The idea of this thesis is to enhance the approximation of reality, when milling and

drilling bone. That is, to develop, implement and validate an algorithm that increases

haptical feedback and interaction when interacting with the specified surgical simulator.

The algorithm that is to be developed, allows various “virtual” points on the shaft of the

surgical instrument(s) to detect collision, “multiple point collision” (MPC) detection. The

feedback is then return as force feedback to a haptic device, which is felt by the user.


6

1.5 Problem Description

There is more than one criterion to this work that must be achieved.

One problem is to develop an algorithm on the basis of patient-specific digital data, like

DICOM-data1, so that the haptic update frequency, 1000 Hz, can be maintained. This

must be achieved while removing material and implement the MPC detection-algorithm,

without impeding the flow and the naturalness of the surgical haptic VR simulator.

Removing material in the simulator is done with a 3D tool image of a pen like instrument

(the stylus), which can mainly drill a hole into a 3D object (a tooth in this case), see

Figure 1.

The simulator has been developed to a stage where the 3D tool is modeled as a sphere

and a cylinder. The placement of the 3D tool in the virtual environment follows directly

the orientation and translation of the tip of the tool on the haptic device [1]. The

simulator, that is, the algorithm only lets you feel the haptics in three ways, xyz-space.

This is a limitation of the haptic feedback. For example, since haptic feedback is felt only

on the tip of the 3D tool, one can not determine by haptics whether the 3D tools shaft

collides with virtual objects or not, see Figure 2.

3-Degrees of Freedom (3-DOF) is a term used to describe this situation. To solve this

problem, development of a 6-DOF feedback in the algorithm is needed, 3-DOF forces

(xyz-directions) and 3-DOF torque (yaw, pitch, and roll).

1 Digital imaging and communication in medicine-data (DICOM-data)


7

Figure 2. The algorithm lets the handle go inside the object.

This is solved with MPC detection algorithm for a 6-DOF haptic device. (Figure by Magnus G. Eriksson [1])

Earlier development of the algorithm lets one “feel” in 3-DOF, xyz-space. This has

previously been satisfying while developing, testing and working with the equipment in

hand.

1.6 Equipment

C++ programming language is used for the low-level programming in a SenseGraphics

AB, H3D API2 scene graph (our virtual environment). The actual scene graph is built up

by scripting languages like Python and X3D. The software lets two different rendering

threads update at 1000 and 30 Hz. The first thread represents the haptic loop and the

second the graphic loop. The graphic loop is then updated and transferred in respective

sample to the haptic loop to provide the force feedback.

The haptic device that produces the haptic feedback in this project is a “PHANTOM®

Omni™” from SensAble Technologies Inc., see Figure 3. To control the haptic device

the OpenHaptics toolkit [16] is used to connect with the scene graph from SenseGraphics.

This device produces only three (x, y, z) ways of feedback, 3-DOF, but delivers 6-DOF

position information (x, y, z, pitch, roll, yaw) from the sensors to the VR simulator. This

2 H3D API is an open source haptic graphic API based on X3D.


8

makes it very difficult to achieve a real life situation. The optimal haptical device for this

algorithm that is to be developed would be one that supports 6-DOF feedback, meaning

forces and torques can be felt by the user. But for many reasons there are only 3-DOF

feedback devices available3.

Figure 3. The Haptic Device, “The PHANTOM® Omni™” from

SensAble Technologies, Inc. (Figure by SensAble [6])

For the verification and validation of the algorithm MATLAB (R2007a) was used to

produce graphs i.e. diagrams. A test sequence was established. During the test sequence

relevant data was logged, the data was then incorporated into MATLAB, which produces

adequate diagrams for analysing and conclusions.

1.7 Delimitations

The MPC haptic algorithm is only tested and based on the current simulator from the

Mechatronics Lab at KTH. This simulator is a prototype and it is constantly used for

3 Further, throughout the project when the MPC algorithm was more or less finished, my supervisor M. Eriksson hired the only 6-DOF device in Sweden, at its time, from colleges in Örebro, Sweden. The project had access to it for 5 days, where this thesis work used it for the verification and validation part of the algorithm.


9

testing and debugging the algorithm, which is presented in this report, throughout the

thesis.

The graphic rendering algorithm is separated from the haptic algorithm and is not edited

in any way, for an eventual improvement. Also the DICOM-files are not altered or

experimented with.

This thesis does not conduct and evaluate an actual test with future users.

1.8 Target Audience

The target audience is mainly researchers and industry entrepreneurs, developing

products and architectures based on haptic simulators. Students and other readers that are

interested in gaining a broad understanding of haptics will also find it valuable reading

this thesis.

1.9 Work Preformed

• An extensive literature study was made during the whole project.

• Setting up, testing and getting familiar with the surgical simulator.

• Interpreting and to understand the current C++ code in the project.

• Discussing different approaches on what methods to be used for the MPC

algorithm.

• Implementing the methods, i.e. write C++ code in the haptic rendering loop.

• Testing, verifying and validating the algorithm with a 6-DOF haptic device.

• Writing and creating this report in English to KTH requirements.


10

1.10 Overview

• Chapter 1 introduces the topic of this thesis and discusses in what situations

haptics can be encountered. The chapter also states the problem in this thesis, the

target audience as well as the delimitations.

• Chapter 2 discusses briefly about similar work done within haptic medical

simulators. It also presents the existing surgical simulator and its haptical

functionalities.

• Chapter 3 presents two different algorithms and its work flow.

• Chapter 4 describes the implementation of the algorithms. Also the hardware

used during development and testing is discussed.

• Chapter 5 goes through the result of the developed algorithms. The algorithms are

verified, analyzed and tested.

• Chapter 6 concludes the thesis and discusses future improvements both for

hardware and software.

Chapter 2 Related Work ________________________________________________________________

11

2 Related Work

There are several studies, works, thesis and scientific articles that explain how haptics

can be used in different fields, e.g. within medicine, and the benefits it has, especially in

the long run both economical and result based, for hospitals and clinics. The importance

of co-located perception in the sense of both visual and haptic spaces is one of the

essential points to perceive a realistic feedback.

Further problems, there are several MPC algorithms today, but they are commonly based

on whether polygonal vertices collide. SmartCollision [17] is an example of a high-speed

collision detection engine for 3D objects. It supports efficient collision detection for

scenes composed of polygon objects, rather than using a voxel4 representation of the

scene. Creating detailed polygonal models of complex organs result in a huge number of

polygons, which increases computation time for collision detection dramatically. Our

simulator uses voxel values to detect collision.

There are also other simulators that use voxel collision detection. Although what these

algorithms are based on is that the sphere, that represents the tip of the surgical

instrument, has multiple points on the surface for collision detection, see Figure 4. This

principle can be used in this algorithm mainly to prevent “fall-through”5 problems. That

is, to prevent falling through holes where the diameter of the hole is smaller than the

diameter of the “sphere-tip” of the instrument [2].

4 A voxel consists of a density value and its coordinates in 3D space. The voxel is created by CT or MRI image processing. 5 This problem has been investigated and partly solved, for the current surgical simulator, by another thesis student (Nazareth Sanchez Rodas, Universidad Politécnica de Valencia, Spain) during the start up (the reading phase) of my thesis work. Unfortunately no documentation about this work has been presented.


12

Figure 4. Multiple points on the “sphere-tip”.

(Figure by Petersik et al. (2003), [3] )

Every collision point returns a force feedback when collision is detected. Calculating

these all together gives us a more smooth and realistic feel when drilling and touching the

object. These algorithm principles can be used for the shaft of the stylus, but instead of

calculating forces, we will calculate the turning momentum (torques) of the stylus.

There is still no documentation found on how the handle of a virtual surgical tool is dealt

with in a haptic VR simulator. Haptic and VR simulation of the bone milling process is a

relatively new and unexplored research area [1].

2.1 The virtual and haptic milling surgery

simulator

A summary of the surgical simulator prototype that is being developed [1] at the

Mechatronics Lab, KTH, will be described in the following sections. Section 2.1.1 and

2.1.2 presents briefly the construction of a realistic visual representation of the milling

process in a VR system. Section 2.1.3 describes how the haptic rendering is preformed

when mill and bone interact.


13

The scene graphs basic function is to present both the visual and physical properties of

the VR environment. All the graphics and haptics are represented in the same scene

graph. This structure is beneficial when additional graphics or force modules are

implemented in the software, where they can share the same data. This enables real-time

interaction between different data in the scene, which is crucial when the surface

structure of bone changes.

The simulator handles the problem of rendering deformable volume data in real time, and

also fulfilling the haptic and VR requirements. The software consists of two different

threads:

• The Graphics loop, updated at 30Hz.

• The Haptic loop, updated at 1000Hz.

Updates from the graphics loop are transferred at each sample to the haptic rendering

loop to provide the force feedback.

2.1.1 Using patient-specific DICOM data

The two most common types of data collection used in medical imaging are based on

computer tomography (CT) and magnetic resonance imaging (MRI) techniques [1]. The

CT scan produces a patient-specific DICOM file, which can be imported in the simulator.

This makes it possible to use the simulator to train for a specific patient before an actual

operation.

Both CT and MRI image processing is used to produce a volumetric model, which can be

regarded as a 3D rectilinear grid of volume elements (voxels) [1]. All kinds of discrete

volumetric data describing the voxel density value of an object can be used by the


14

simulator. The only needed information is the density value of a certain voxel at a known

position in 3D space.

The data from the DICOM file are implemented in a three-dimensional data matrix that

stores all the density values of the voxels. Both the graphic rendering of an object and the

haptic force feedback rendering use this data matrix during interaction.

2.1.2 Graphic rendering

The Marching cubes algorithm [18] is used for the surface-rendering. It is an efficient

method using voxel density values to perform a high-quality visualization of the surface.

The VR representation of the 3D mill and bone interaction process consists of the

following elements [1], which are described briefly:

1. Read the converted volumetric DICOM data. Store the data in a 3D matrix. Create

3D matrices representing the voxel gradients based on the density values (this is

done only once, on start-up).

2. Create an octree node structure based on a predefined tree depth value. Set the

range of coordinates and min/max density values in every node (this is done only

once, on start-up).

3. Check for the milling; if there is milling, then update the min/max density values

and the gradient values in the node structure (looped).

4. Apply the Marching cubes algorithm to the updated tree nodes. Create vectors of

the normals and vertices of the surface elements (triangles) to be visualized

(looped).

5. Render the created vectors of the normals and vertices that represent the triangles

used to model the shape of the object (looped).


15

2.1.3 Haptic rendering

The haptic rendering is based on the voxel information taken from the CT data, which

contains both the voxel density value and the voxel position in space. The graphic and the

haptic rendering loop processes are only connected to each other by the same data matrix

containing the voxel density values and the voxel coordinates. The haptic loop has the

following procedures:

• Calculate the density value at the center position of the sphere representing the

mill.

• If this density value is larger than the isovalue, collision is detected and the

proxy-probe-based method (see Section 1.3) is run. If not, there is no collision

and the proxy position follows the position of the probe.

• Calculate the force using this equation:

force = k * (proxy_position – probe_position).

• If milling, add vibration force.

• Return the total force to the haptic device.

After collision is detected in the haptic loop, a force will be sent to the haptic device. The

direction and the magnitude of the force is based on the proxy-probe distance

relationship. The proxy (the virtual tip) is always on the surface while the probe

corresponds to the real position of the device. The idea of the algorithm is to calculate the

motion of the proxy on the “surface”, based on the difference between the position of the

proxy and the probe.

By first moving the proxy along the tangential plane of the surface, also checking

whether the new proxy position is inside or outside the surface, and then determining the

point of intersection with the surface in order to determine the new position of the proxy.

The calculated haptic force is proportional to the distance between the probe and the new

proxy position.


16

M. Eriksson [1] describes the haptic rendering algorithm more mathematically detailed in

his licentiate thesis.

Chapter 3 The Algorithms ________________________________________________________________

17

3 The Algorithms

As there was no previous work found on how the handle of a stylus is dealt with. The

methods chosen are partly inspired from the previous work [1] done by PhD candidate M.

G. Eriksson. In this section of the paper we present two different MPC algorithms that

were developed.

3.1 The Line Approximation algorithm

Essentially the idea of this algorithm is inherited from the previous work done in this

project [1]. That is, using a proxy-probe based method6. This is used mainly to avoid

haptic fall-through problems, when the user pushes the haptic device into an object and

the 3D tool (actually the algorithm) does not compensate to stay on the surface of the

object. This proxy-probe method also allows the use of spring models as force algorithms

and permits the user to drag and push the tool along the surface of the object. To model

frictional surfaces, to “feel” friction against an object, the proxy-probe method is

beneficial.

3.1.1 Discretization of the tool

The approach of how to handle this problem came very natural as it was inspired from M.

Eriksson’s work. The tool is represented as a sphere and a cylinder with respective

proportional dimensions on the screen. In the haptic MPC algorithm the tool is

represented as a centerline that consists of discrete vertices for both collision detection

6 Described previous in this paper, see Definitions.


18

and calculation of the forces and torque feedback. The cylinder is approximated with an

arbitrary amount7 of vertices along the centerline and the sphere is approximated with

one vertex in the center of the sphere, see Figure 5.

Figure 5. Discretization of the milling tool.

3.1.2 Collision Detection

The collision detection between the tool and the object is accomplished by checking the

density value, voxel8, of each vertex placed along the centerline of the cylinder. The

density value is then compared with the predefined isovalue9, which is taken mainly from

CT-scans10.

There are two cases when applying collision detection:

7 So that the computational load will not increase and affect the haptic rendering rate (1000 Hz), when calculating. 8 Each voxel in a 3D volumetric dataset consists of a density value in the 0-255 range (8-bit) [1] 9 Indicates the density value level of the voxels attenuation values done by the graphic rendering of the 3D

object. 10 When representation of hard and stiff objects, such as bone, is needed.


19

1. If all of the voxel values of the vertices are less than the isovalue: Then no

collision has occurred. Neither force nor torque feedback is calculated. The

position and orientation of the tool is saved to the next haptic loop at 1000Hz.

2. If anyone of the voxel values of the tool exceed the current isovalue: Then there is

collision between the tool and the object. If the center point of the sphere also has

collision then forces feedback will be calculated based on previous research [1].

The torque feedback will be calculated, only if anyone of the discrete centerline

vertices in the cylinder has collision, before sending the signal to the haptic

device. Calculations of the torque will be presented later on in this report.

To explain further more regarding the collision detection procedure, one could say, that

between the different times t (the current haptic rendering loop) and t-1 (the previous

haptic rendering loop) there is a difference in the volumetric density value. Not only is

the collision detected procedure implemented when there are volumetric differences, but

also other calculations are made before returning the adequate feedback to the haptic

device. The collision detection and all the calculations are made within the same haptic

rendering loop, i.e. the haptic loop rate is 1000Hz.

3.1.3 Find Intersection Point

When collision is detected between any of the centerline vertices of the tool, an

intersection point is determined. The intersection point will represent the collision point

itself, i.e. the first vertex of the centerline of the tool that enters the surface of the object,

see Figure 6. The orientation difference, α , and the translation vector, d, from time step

t-1 to t is calculated.


20

There are two main states when finding the intersection point (IntP). Either there was

collision at time t-1 or there wasn’t collision at t-1, as in Figure 6.

Figure 6. The centerline is translated (d ) and rotated (α ) from t-1 to t.

In this case there is no collision at t-1 but collision at t. The cross marks the intersection point.

3.1.3.1 If no collision at time step t-1

In this state (no collision at time step t-1 but collision at time step t) the IntP can be

identified by using a fraction based method, which will be presented further on in this

section. Initially we have to ascertain which of the vertices that are “active” vertices. This

is done by applying two different methods, translation/rotation method and a ray-casting

[13] method.


21

Basically the idea is to go step by step, by a predefined fraction size, from time step t-1 to

t, until we find the collision for any of the vertices of the centerline. The vertex or the

vertices that have collision at the fraction step n are called “active vertices”. We also save

the position of the active vertex or vertices at fraction step n-1 (before collision) for

calculating and deriving the IntP, further on in this section.

If there is more than one active vertex, see Figure 7, then they are compared to each other

by using the density value fraction equation (Eq.1). The index n-1 indicates the step

before any collision and the index n indicates the current11 fraction step. Notice that this

procedure is done both in the translation/rotation method and in the ray-casting method.

Figure 7. The Active vertices and the most active vertex, which are presented in fraction

steps n-1 and n at time step t. (Figure by Magnus G. Eriksson)

nn

n

valuevalue

valueisovalueFraction

−

−=

−

−

1

1 _ (Eq.1)

11 If collision detected on one or more of the vertices of the centerline, then there will not be any more

fraction steps, i.e. n+1 etc. Hence it is called the “current” fraction step.


22

The active vertex that has the smallest fraction value, see Figure 8, is the most active

vertex, i.e. the one that gives us the correct IntP (Eq.2).

Figure 8. The smallest fraction.

The cross marks the intersection point, IntP

minmin1 )1( FractionVertexmostActiveFractionVertexmostActiveIntP nn ⋅+−⋅=−

(Eq.2)

Notice when searching for the active vertices, using both the translation/rotation and ray-

casting method, is that we go from t-1 to t and not the other way around i.e. from t to t-1.

The fraction step, n, starts from t-1 and moves towards t fractionally until we detect

collision. If there is no collision found in fraction step n then we continue with the next

fraction step. The fraction step n that did not have collision will be our n-1 in the next

fraction step, and the new fraction step, n, is checked for collision. When collision is

detected we stop with the fraction stepping and determine which vertices are the active

vertices.

If we were to go from t to t-1 and stop at the first detected collision it would not

necessarily mean that this is the IntP that we are looking for. One needs to go the whole


23

trajectory from t to t-1 otherwise the surface with the actual IntP is missed, see Figure 9.

This means that the search algorithm for the active vertices is more efficient when going

from t-1 to t.

Figure 9. The inefficient way of searching for the active vertices.

The blue lines represent the tool. The red dots represent vertices with collision. The green dots represent vertices with no collision.

3.1.3.1.1 Ascertain Active Vertices by Translation/Rotation method

First out, we check for collision for the vertex that belongs to the tip of the tool, i.e. the

center point of the sphere. If there is collision the proxy is calculated so that it remains on

the surface of the object. The algorithm that is used for this was developed earlier in this

project [1], before this thesis. As the Figure 10 presents, the discrete centerline is

translated out as an offset to this proxy position. This translation will be the foundation

for the calculated haptic feedback, which will be presented later on.


24

Figure 10. Illustrating the translation/rotation algorithm.

The new position of the discrete centerline is marked as offsett , see Figure 10. Both the

rotation (α ) and the translation ( d ) difference of the discrete centerline from t-1 to offsett ,

are easily derived and established in the algorithm. The number_ of_ rotation_ steps and

number_of_translation_steps are predetermined constants (i.e. the fraction steps).

Consequently, the rotation step size, α∆ , and translation step size, d∆ , are determined as:


25

STEPSROTATIONOFNUMBER ___

αα =∆ (Eq.3)

STEPSNTRANSLATIOOFNUMBER

dd

___=∆ (Eq.4)

Now that we elucidated the step procedure, the algorithm starts with the translation

sequence. The discrete centerline is moved step by step d∆ from time step t-1 towards

offsett . At every step, n, the density value of each vertex is calculated. If no collision is

detected when translated all the way to d the rotation sequence of the algorithm is

started, see Figure 10. Now the discrete centerline is rotated step by step α∆ until

collision is detected by any vertex/vertices.

When collision is detected (within the translation or rotation sequence) during step n, the

IntP is determined by knowing which vertices are active. How to determine the IntP is

explained in the previous section above (“If no collision at time step t-1”).

3.1.3.1.2 Ascertain Active Vertices by Ray Casting method

Both the translation/rotation and the ray casting method are based on the same principle,

i.e. the fraction stepping. The difference is how the algorithm searches for the active

vertices.

In the ray casting algorithm, each vertex at t-1 casts a ray, ia , to respective vertex at offsett ,

see Figure 11. The number_of_raycasting_steps is a predetermined constant.

Consequently, the ray casting step size for each vertex, ia∆ , is determined as:

STEPSRAYCASTINGOFNUMBER

aa i

i___

=∆ (Eq.5)


26

Figure 11. Illustrating the ray casting algorithm. (Figure by Magnus G. Eriksson)

The search algorithm starts by translating each vertex with the specific step size, ia∆ ,

along the discrete centerline. The density value of each vertex is calculated at every step

(i.e. every fraction step). If no collision is detected at a specific step, the vertices along

the discrete centerline are translated towards offsett until collision of any vertex or vertices

are found. When collision is detected during fraction step n, the IntP is determined by

knowing which vertices are active. The coordinates and density values of the active

vertices from fraction step t-1 and t are used to acquire the intersection point.


27

3.1.3.2 If collision at time step t-1

The following explanation of the algorithm is presented together with Figure 12. Parts

from earlier developed proxy-probe algorithm [1] will be used when finding the

intersection point at time step t if there was collision at time step t-1. The main part of the

algorithm that is used is the movement of the proxy on the surface based on, the relative

position to the “probe of that point” and the friction of the surface.

When there was collision at time step t-1, the intersection point ( 1−tIntP ) is known and

also the position of the tip of the tool is known, i.e. the center point of the sphere ( tCP ),

when the current haptic loop starts (time step t). Knowing these to points we derive a

vector, b , between these two points. The discrete center line with the vertices is projected

along this vector. Each vertex has its density value calculated and the one with the

maximum density value is stored. This vertex with the maximum density value ( maxv ) is

the one that is located deepest into the object. It is used as “the probe” in the algorithm

that is mentioned above in this section. The proxy is the 1−tIntP in the beginning of that

algorithm. Then the algorithm calculates the new position of the intersection point

( tIntP ), which is our new proxy position.


28

Figure 12. Showing the procedure when collision at time step t-1.

The marked crosses represent the intersection points. (Figure by Magnus G. Eriksson)

3.1.4 Calculate the haptic Force- and Torque feedback

When collision occurs in the virtual environment, then there are two different haptic

forces calculated. The explanation of the force algorithm is presented in conjunction with

Figure 13.

1. When the center point of the sphere ( POINTprobe ) collides with a surface of an object,

the force that is calculated is based on earlier developed spring-model algorithm [1].

The algorithm determines the POINTproxy and a vector, e , is derived (Eq.6) from the

know position of the proxy and the probe.


29

POINTPOINT probeproxye −= (Eq.6)

Vector e represents the springs length and direction. Therefore, the haptic force

feedback ( proxyPOINTF ) at the POINTproxy is calculated as:

ekF springproxyPOINT ⋅= (Eq.7)

The spring constant, springk , is set to a high value when colliding with stiff materials.

2. The torque (T ) force around the POINTproxy based on where the intersection point is

on the tools shafts.

When the intersection point is once established, a proxy for the discrete centerline of

the tools shaft ( shaftproxy ) is placed in the direction from the POINTproxy to the

current intersection point ( tIntP ). A vector, r , is defined (Eq.8) between these two

points.

POINTt proxyIntPr −= (Eq.8)

To calculate the force at the intersection point ( IntPF ) which in turn gives us the

torque (T ), we need to establish the corresponding intersection point ( probeShaftIntP )

on the shaft of the probe (Eq.9).

The position of the discrete centerline of the probe ( shaftprobe ) is know, because it is

the same position of the real physical shaft (the haptic device sends the coordinates to

the simulator). Hence, the direction of shaftprobe is known ( ionShaftRotat ).

ionShaftRotatrIntPprobeShaft ⋅= (Eq.9)

The reacting force is based on the earlier developed spring model algorithm[1], where


30

the spring length and the direction ( g ) is calculated as following:

probeShaftt IntPIntPg −= (Eq.10)

The reacting force at the intersection point is:

gkF springIntP ⋅= (Eq.11)

The torque that one “feels” with the haptic device when the tools shaft collides with

an object is calculated as (notice the cross product):

IntPFrT ×= (Eq.12)

The torque, T , and the force, proxyPOINTF , are then return to the haptic device for a

realistic 6-DOF feedback.


31

Figure 13. Illustrating the Torque (T ) and Force ( intproxyPoF ) calculations.

3.1.5 Placement of the Proxy for correct visualization

In the algorithm describe above (The Line Approximation), the cylinder that represents

the shaft of the tool has been approximated with a discrete centerline. The discrete

centerline follows directly the translation and orientation of the physical tool of the haptic

device. When the above mentioned calculations are done of the discrete centerline the

cylinder in the scene graph, i.e. in our virtual environment, would be a radius distance


32

inside of the collided object. Therefore, there must be an offset of the shaftproxy so that

the cylinder in the scene graph is correctly situated. To determine this offset, the principle

of a method used in earlier development [1] is applied.

In earlier development the method moves out the POINTproxy , i.e. the center point of the

sphere that represents the tip of the tool, an offset ( tPOINTOffseproxy ) for correct

visualization.

The calculations of the offset of the proxy cylinder, tshaftOffseproxy , is illustrated in Figure

14. The offset of the intersection point ( tOffsetIntP ) is calculated firstly. This is achieved by

using the gradient ( u)

) of the current intersection point ( tIntP ) and the radius of the

cylinder.

radiusuIntPIntP ttOffset ⋅+= ˆ (Eq.13)

The vector for the discrete centerline of the cylinder that is visualized is calculated as the

following:

tPOINTOffsetOffsettshaftOffse proxyIntPproxy −= (Eq.14)


33

Figure 14. Shows in conjunction with the text above how the offset of the cylinder is

handled.

3.2 The Surface Approximation Algorithm

This section describes the second developed force and torque algorithm. The main

difference from “The Line Approximation” algorithm is that it is not proxy-probe-based.

This means that, if the user pushes the tool against the surface of the object harder than


34

the force or torque that is being produced by the haptic device, then the representation (in

the scene graph) of the tool will be put into the object, i.e. it will not stay on the surface

as in the proxy-probe-based method. Another difference is that it is not possible to model

any frictions when dragging the tool along the surface of the object.

The fundamental idea of this algorithm is to put vertices on the surface of the tool, both

on the sphere12 and the cylinder. These vertices are used for collision detection and

calculations of the force and torque feedback. The movements, i.e. translation and

orientation, of the physical tool of the haptic device are represented directly in the virtual

environment.

When comparing the two algorithms, this one has the advantage of returning a more

stable haptic feedback because of the force summation of all the vertices that collides

with the object. The unstableness of the haptic feedback in “The Line Approximation”

algorithm can also depend on when moving out the proxy a radius in the gradient

direction, solely because of this gradient direction is based on one single point (the

intersection point). In this second algorithm this problem is not encountered.

3.2.1 Discretization of the tool

Just as the previous algorithm, we start with the discretization of the tool with vertices for

both the calculations of the haptic force and torque feedback, and also for the collision

detection. The 3D tool is built up by a sphere and a cylinder, which represents the tip

respectively the shaft of the tool. The sphere is approximated with vertices around the

surface and one vertex in the center. The cylinder is approximated with multiple slices

where one vertex is put in the center of each slice and vertices along the circumference,

i.e. the vertices are put on the surface of the cylinder in layers, see Figure 15.

12 Putting multiple vertices on the sphere and preventing “fall-through” has been done by another master thesis student, Nazareth Sanchez Rodas, Universidad Politécnica de Valencia, Spain.


35

The amount of vertices on the exposed surfaces and the amount of slices can easily be

changed in our developed algorithm. Nevertheless, during the test and verification phase

the suggested number of slices and vertices on the cylinder is 20 respectively 9 each. For

the sphere it is suggested 26 vertices around the surface (see Section 5.2.2). These

numbers gives us the best performance to fulfill the real time demands of the simulator

when finding collision of a surface and calculating the haptic force and torque feedback

(see figure 28 in Section 5.2.2).

Figure 15. Discretization of the milling tool.

3.2.2 Collision Detection

As in the previous algorithm the collision detection is preformed by comparing the

density values between the tool and the object. At each time step t, the vertices are put

out on the tool at the specific positions and checks if the voxel is higher than the

predefined iso-values, which represent the density of the object. There are three cases

when applying collision detection:


36

1. If all of the voxel values of the vertices are lower than the isovalue: Then no

collision has occurred. Neither force nor torque feedback is calculated. The

position and orientation of the tool is saved to the next haptic loop (i.e. the next

time step t) at 1000Hz.

2. If the voxel value for the either the center point of the sphere or a particular slice

is higher than the iso-value: Then there is collision between the tool and object.

The reaction force for that specific slice or the haptic force feedback will be the

same as the time step before the current one, i.e. the same as the time step at t-1.

This will be explained further on in the next section.

3. If the voxel values of the peripherical vertices are higher than the iso-value: There

is collision between the tool and object. How the force and torque feedback is

calculated is described in the next section below.

3.2.3 Calculate the haptic Force- and Torque feedback

We start looking at the haptic force feedback that is produced when only the peripherical

vertex of the sphere (i.e. tip of the tool) collides with the object. This method is also

applied for the force calculations of each slice on the shaft of the tool. The force that is

generated is the sum of all vertices that have collision, see Figure 16.

A spring-model is implemented (Eq.15) to calculate the force for each vertex that has

collision.

iparameterresolutionspringivertex akkF ⋅⋅= __ (Eq.15)

Where ia is the vector from the vertex to the intersection point with the surface of the

object. Its direction is towards the center point of the sphere. springk is the spring constant


37

and parameterresolutionk _ is a constant used to weigh up the number of vertices concerned in

the present collision (Eq.16).

collisionpresentinverticesofamount

verticesofamounttotalk parameterresolution

_____

____ = (Eq.16)

The total force on the sphere is set by summarizing the forces from each vertex (Figure

16 is in 2D , for the sphere the vertices are spread in 3D):

∑=

=

=

ni

iivertexspheretotal FF

0__ (Eq.17)

Where n is the number of concerned vertices in present collision.

Figure 16. Illustrating the calculation of force feedback for the sphere in 2D or

the calculation of the reaction force for each slice that has collision. (Figure by Magnus G. Eriksson)


38

The second case of collision, when the center point of the sphere has collision – then the

force feedback that is sent to the haptic device at time step t is the same as the force

feedback at time step t-1.

1____ −= tspheretotaltspheretotal FF (Eq.18)

To motivate Eq.18, one has to assume that more than half of the sphere has collision and

that we would apply the previous force algorithm, Eq.15 & Eq.17, on that case. Doing

this the force summation could be zero or that the force could result in the wrong

direction, see Figure 17. When applying the Eq.18 on this case we get a visual error,

because the 3D tool would find itself inside of the manipulated object, which is

physically impossible.

But, the goal of this algorithm is to maintain the haptic realness. So the Eq.18

approximates us to that realness by wanting to push the tool back out again from the same

direction it initially came from. The same case, equation and motivation are also used on

the reaction force for each slice that is projected on the shaft (Eq.19).


39

Figure 17. The force summation can result in zero when the complete sphere or

a slice of the shaft is inside the object.

1______ −= tjslicereactiontjslicereaction FF (Eq.19)

When collision is detected in the cylinder, i.e. the shaft of the tool, a haptic torque

feedback, totalT , is produce on the haptic device. How this is achieved is described below,

see Figure 18.

The reaction force for each slice is calculated by using the same method as the Eq.17

above.

∑=

=

=

ni

iivertexjslicereaction FF

0___ (Eq.19)

A torque, jsliceT _ , around the tip of the tool( i.e. the center of the sphere) is calculated

derived from the respective force reaction of each slice.


40

jslicereactionjjslice FrT ___ ×= (Eq.20)

Where jr is the vector from the tip of the tool to the center point of the slice j. By

summarizing all the torques from each slice gives us the total torque.

∑=

=

=

mj

jjslicetotal TT

0_ (Eq.21)

The force, spheretotalF _ , and the torque, totalT , are then returned to the haptic device for a

realistic 6-DOF feedback.

Figure 18. Illustrating two view-planes of the reaction force for a slice with collision and

its torque. (Figure by Magnus G. Eriksson)

Chapter 4 Implementation ________________________________________________________________

41

4 Implementation

This section will describe briefly how the method is implemented into the software so

that the corresponding hardware would work properly.

4.1 Workstation

The hardware of the workstation differed depending in which point of the process of the

development one was working with. The process can be split into two cases:

1. Integration & implementation

2. Testing & Verifying

The main difference between these two cases is the use of two different haptic devices.

4.1.1 Integration & implementation

During the beginning of the development process, the PC used for the simulator was

equipped with an Intel Core 2 Duo processor E6600 – 2.4 GHz with an nVidia Quadro

FX1400 graphics card.

The haptic device was a PHANTOM® Omni™, see Figure 3. This device has a 6

degrees-of-freedom positional sensing but can only produce a 3-DOF feedback. The

haptic maximum force that it can produce (in 3-DOF) at nominal position is 3.3 Newton


42

[21]. The workspace of the Omni is enough to mimic a real surgical situation [1]. The

PHANTOM Omni model is the most cost-effective haptic device available today [21].

4.1.2 Testing & verifying

During the second phase of the development process during this thesis, we moved the

simulation platform to a computer which theoretically has better performance capabilities

for best test results. The PC was equipped with a Pentium 4, 3.3 GHz dual processor with

an nVidia Quadro FX1400 graphics card.

The haptic device was a PHANTOM Premium1.5/6DOF from SensAble13, see Figure 19.

This device has a 6-DOF input and output which is the optimal for our purpose. The

haptic maximum force that it can produce (in 3-DOF) at nominal position is 8.5 Newton

[22]. The workspace of the Premium is enough for our simulations. One inconvenience

with the haptic device is the poor stiffness because of the structure of serial linked arms.

This is very important for a realistic feeling when interacting with stiff material such as

bone.

13 This haptic device was leased for a week from colleges/ associates in Örebro.


43

Figure 19. The PHANTOM ® Premium 1.5/6DOF from SensAble.(Figure by SensAble

[6])

4.2 Software

The developed haptic milling surgery simulator is run in a H3DAPI [5] scene graph,

which is a software platform for multi-sensory applications. The H3DAPI scene graph

communicates with the OpenHaptics toolkit [6] for control of the haptic device. The C++

programming language is used for low-level programming in the H3DAPI and it is here

where we will integrate and test our algorithms.

To build up the scene graph we use the scripting language X3D to our help, which makes

it fairly easy to change the scene without the need of compiling the application. A

modified version of the Marching Cubes [1] algorithm is used for 3D rendering of the

milling process. The 3D graphic rendering rate is at 30Hz and the haptic rendering rate is

updated at 1000Hz.

A high-resolution DICOM CT dataset is used to build up the data, i.e. the voxel value, of

our object that is to be manipulated (in this case a tooth).


44

Independently from manipulating the VR representation of the object by milling or

material removal, the simulator also allows the use of cut planes, zoom and rotation to

examine interesting parts of the data.

Sound effects are also implemented to make the simulation when milling more realistic

[1]. The developed haptic milling surgery simulator can also be used for other

applications, such free form sculpting or dental training simulations

.

Chapter 5 Results ________________________________________________________________

45

5 Results

In this chapter the algorithms described in the chapter 3 (The Algorithms) are verified,

analyzed and tested in the simulator. The first part presents the verification and analysis

of the two algorithms. The last part presents a testing procedure for optimal performance.

All the data was logged to a text file and were processed in MATLAB to produce the

diagrams below. For a visual experience of the algorithms, there are two short videos on

the homepage of the Haptic Simulation Surgery Project [8].

5.1 Verification and analysis

The verification procedure of the two algorithms is performed by moving the virtual tool

in a predetermined plan. The virtual environment consists of the milling tool (the cylinder

& the sphere) and a 3D cube, see Figure 20. The cube is created in the X3D scene script

and the discrete density values are build up the same way as the medical objects.

Chapter 5 Results ________________________________________________________________

46

Figure 20. The virtual environment used for verification.

The verification sequence for the two algorithms is performed as follows. The user moves

the virtual tool until the tip collides with the object (the cube), then starts milling a hole,

i.e. removing material. When the tools tip and an arbitrary length of the shaft (N.B. not

the whole shaft) is inside the hole, the user moves the shaft so it collides with the edge of

the milled hole.

For the “Line approximation” algorithm the following data is logged during the

verification procedure:

• The haptic rendering rate

• The lever arm

• The spring length

• The reaction force

• The torque

Chapter 5 Results ________________________________________________________________

47

The data that was logged for the “Surface approximation” algorithm were:

• The haptic rendering rate

• The lever arm for each slice that collides

• The reaction force for each slice that collides

• The torque for each slice that collides

• The total torque

(For description of the parameters see Chapter 3, The Algorithms.)

The results and analysis of the logged data is presented in the following sections below.

The material of the cube is assumed to have high stiffness (high spring constant) and low

friction.

5.1.1 Evaluation of the Line Approximation Algorithm

Looking at Figure 21 one could easily see that the average haptic rendering rate during

the test for the “Line approximation” algorithm is in between 950-1000 Hz, which is

satisfying for the real-time requirements. The amount of vertices chosen along the

centerline of the cylinder is presented in the section 5.2.

Chapter 5 Results ________________________________________________________________

48

Figure 21. The logged haptic rendering rate during collision of the shaft

against the edge of the hole (Line).

The next graphs see Figure 22, shows that the algorithm calculates the correct torque

value as the cross product of the lever arm and the reaction force (Eq.12, Section 3.1.4 ).

This is evident while the torque curve (lower right graph in Figure 22) directly follows

the curves of the lever arm (lower left graph in Figure 22) and the reaction force (upper

right graph in Figure 22). The reaction force in turn is based on a spring model (Eq.11,

Section 3.1.4), which can be seen by comparing the two graphs (upper left vs. upper right

graphs in Figure 22) for these parameters.

Chapter 5 Results ________________________________________________________________

49

Figure 22. The logged data of the spring length (upper left), force reaction (upper right),

lever arm (lower left) and the torque (lower right).

While performing the verification session some instability and vibrations are felt when

holding the haptic device. The graphs in Figure 22 clearly reveal this problem of

vacillation and disturbance.

One solution is to implement a filter in the algorithm which would make the signal more

stable, thence reducing this fluctuation.

But, logically this is what happens when two stiff materials collide; one could compare

the collision situation with the brakes of a car, i.e. the brake blocks colliding with the

wheel. Vibrations occur and the magnitude is based on how hard the user is holding and

grasping the haptic device.

Chapter 5 Results ________________________________________________________________

50

In spite of this disturbance the intersection point is found at correct position (see section

5.2). Also earlier research [1] verifies that collision detection works well and the proxy

directly follows the surface of the object when dragging the tool along the side of the

object.

5.1.2 Evaluation of the Surface Approximation Algorithm

The verification session begins with examining the haptic rendering rate, see Figure 23.

The haptic rate is an issue for the Surface algorithm. There needs to be a sufficient

amount of slices and vertices on the surface of the tool, so correct collision detection can

be achieved without missing any part of the object. Therefore, if the number of slices and

vertices are too high, the required haptic rate can not be satisfied and stability problems

occur when the two stiff materials collide. Nevertheless, the verification test argues that

the haptic rate shows satisfying values and the requirements are fulfilled. The right

amount of slices and vertices for optimal performance are presented in section 5.2.

Chapter 5 Results ________________________________________________________________

51

Figure 23. The logged haptic rendering rate during collision of the shaft

against the edge of the hole (Surface).

In Figure 24 the graphs show data from four sequential arbitrary slices during the

verification test. The lever arm graph indicates that the distance from the center of the

sphere to each of the slices that has collision is calculated correct. Considering the

reaction force of each slice it shows that it is proportional to the distance for the specific

slice. The slice with a larger lever arm is pushed longer into the object and therefore

gives a bigger force reaction, this also applies the other way around, i.e. the slice with a

shorter lever arm is pushed only slightly into the material and hence gives a force that is

smaller, which is logical. The torque for each slice is the cross product between the

specific lever arm and the respective reaction force (Eq.20). According to the lower left

graph in Figure 24 the torque for each slice follows directly the reaction forces. The total

torque is the vector sum of the torques for each slice (Eq.21). Analyzing the Surface

algorithm, with the graphs in Figure 24 (especially the lower right total torque graph) we

Chapter 5 Results ________________________________________________________________

52

can clearly see that the disturbance is less than in the Line algorithm. When the user is

holding and using the haptic device during the verification sequence the reduction of the

disturbance is apparent. All the same, implementing a filter in the Surface algorithm

would increase the stability even more.

Figure 24. The logged data of the lever arm for each slice (upper left), the reaction force

for each slice (upper right), the torque for each slice (lower left) and the total torque (lower right) for the Surface approximation algorithm

5.2 Test Application

To determine the optimal amount of vertices along the center line in the Line

approximation algorithm and the amount of slices and vertices in the Surface algorithm a

test application was created.

Chapter 5 Results ________________________________________________________________

53

The virtual scene involves the milling tool and a 3D representation of a tooth taken from

a CT-image with the resolution of 256 x 256 x 176. Besides the virtual objects in the

environment, see Figure 25, the red sphere in the figure shows the correct position of the

intersection point (IntP) using the Line approximation algorithm.

Figure 25. The virtual scene for the test application. The red sphere shows the position of

the intersection point using the Line approximation algorithm.

During the test the performance of the translation/rotation and ray casting techniques in

the Line approximation algorithm is studied.

The tests show that the real time requirement of the haptic rate at 1000Hz must be

achieved by 95% (950Hz) otherwise the stability problem of the haptic device makes the

vibrations insupportable when the two stiff materials collide. Because of the collision

between two stiff materials there tends to be a fluctuation in the haptic force feedback.

Also to keep in mind is when logging relevant data to a file during the test sequence the

haptic rendering rate will be affected.

Chapter 5 Results ________________________________________________________________

54

Therefore, the investigation is executed by comparing the amount of vertices and the

haptic rendering rate to each other.

5.2.1 Test results for the Line approximation algorithm

The figures below show the result of the test sequence done for the Line approximation

algorithm. The material of the tooth is assumed to have high stiffness (i.e. a high spring

constant) and low friction.

In Figure 26 the dashed lines represent the average values, from top to bottom, of the

haptic rendering rate, which are:

• 50 vertices: 984 Hz





Chapter 5 Results ________________________________________________________________

55

Figure 26. The haptic rendering rate for different amount of vertices

for the Line approximation algorithm.

The analysis shows us that 50 vertices along the center line gives the best real-time

requirements and still have good precision of the multi point collision detection.

The two methods, the translation/rotation and ray casting methods, are also compared to

each other, to find out which one suits the best for this algorithm. The amount of vertices

chosen is, 50. In Figure 27 the dashed lines represent the average values of the haptic

rendering rate, which are:

• Translation/Rotation method: 972 Hz

• Ray casting method: 973 Hz

Chapter 5 Results ________________________________________________________________

56

Figure 27. The haptic rendering rate for the translation/rotation and

the ray casting method for the Line approximation algorithm.

Thus, the conclusion is that it does not matter which one of the methods is used to find

the intersection point (IntP) in the Line approximation algorithm.

5.2.2 Test results for the Surface approximation algorithm

For this test a fixed amount of vertices around the sphere and around each slice is chosen,

see Figure 28. There will always be one vertex in the center of the sphere and each slice.

The amount that is chosen is based on the minimum amount of vertices that gives a

satisfying haptic feedback and correct collision detection. The resolution is:

• Amount of vertices on the surface of the sphere: 26

• Amount of vertices on the circumference of the slice: 8

Chapter 5 Results ________________________________________________________________

57

Figure 28. The position of the vertices on the slices circumference and

the surface of the sphere.

By determining these numbers, the tests consist in finding the optimal quantity of slices

to best fulfill the real-time requirements, and of course the best haptic feedback. In

Figure 29 the dashed lines (from top to bottom) represent the average values of the haptic

rendering rate, which are:

• 10 slices: 974 Hz





Chapter 5 Results ________________________________________________________________

58

Figure 29. Haptic rendering rate for the different amount of slices

for the Surface approximation algorithm.

The analysis shows us that 20 slices gives the best real-time requirements and still have

good precision of the multi point collision detection.

To conclude the test, the following statements can be made from the analysis:

• For the Line approximation algorithm the total sum of vertices is 51, i.e. 50

vertices on the centerline of the cylinder and 1 for the sphere. It does not matter if

the translation/rotation or ray casting method is used to find the intersection point.

• For the Surface approximation algorithm the total sum of vertices is 207, i.e. 26

vertices on the surface of the sphere, 1 vertex for the center of the sphere, 8

vertices on the circumference of each slice and 1 vertex for the center of each

slice. The mathematical summation would be, (26 + 1) + ((8+1)*20) = 207.

Chapter 6 Conclusion ________________________________________________________________

59

6 Conclusion

We start by returning to the purpose and goal of this thesis. The main objective was to

further develop and enhance the existing haptic algorithm in the medical surgery

simulator for the Mechatronics Labs at the KTH. It also hade to meet some requirements

like maintaining the haptic rendering rate (1000 Hz) while removing material and the

implementation of the MPC detection. Could this be achieved? The answer is yes. We

have come closer to emulate reality. Even though there are some disturbances in the

application, the results are satisfying.

6.1 Discussion

Using different approaches we could learn and understand what works, what works good,

what works better and which parts of a whole haptic simulator one must consider to

achieve a haptical improvement.

Depending what kind of application the simulator is for each algorithm presented in this

paper has its advantages and disadvantages:

• The Line Approximation algorithm has friction in consideration but less accuracy

in the haptic force feedback than The Surface Approximation algorithm.

• The Surface Approximation algorithm has no friction in consideration but higher

accuracy in the haptic force feedback.

However, the investigation in the haptic field is still at an infant stage and there is much

more to explore. As the improvement of hardware is continuous, the more complex the

calculations will be, but also more accurate. E.g. the haptic force model used today is a

Chapter 6 Conclusion ________________________________________________________________

60

simple spring model. Force feedback that occurs in reality is much more complex. One

should consider the velocity of impact when the tool collides with the object, dimensions

and rotational speeds when milling, and material data regarding the objects, all these are

important parameters for a realistic force model [1]. But incorporating all these

parameters would be expensive for today’s hardware and the real time requirement would

be difficult to attain. The problem of emulating reality is not only applied for the capacity

of today’s computers, the haptic devices are as equally important in this kind of

simulators. The lack of stability, especially for the haptic devices that are built with serial

arms (as the Phantoms, see Figure 3), is a substantial factor when reproducing haptic

force feedback. Haptic devices that are constructed with parallel motors have a more

stable feedback14, as the Novint Falcon [7] that was released in the autumn 2007,

although the Falcon only works in 3-DOF.

A 6-DOF device that has parallel motors should be the next device to revolutionize the

haptic market in the near future.

14 This knowledge is attained by self experience, i.e. from testing, using and comparing the haptic

equipment.

References ________________________________________________________________

61

References [1] Eriksson, Magnus G., Haptic and Visual Simulation of a Material Cutting

Process, Licentiate Thesis in Technology and Health, 2006 [2] Petersik, B. Pflesser, U. Tiede, K.H. Höhne, R. Leuwer, Realistic haptic volume

interaction for petrous bone surgery simulation, University Hospital Hamburg-Eppendorf, Germany, 2002

[3] Andreas Petersik, Bernhard Pflesser, Ulf Tiede, Karl-Heinz Höhne, and Rudolf

Leuwer, Realistic Haptic Interaction in Volume Sculpting for Surgery Simulation, University Hospital Hamburg-Eppendorf, Germany, 2003

[4] Haptics, http://en.wikipedia.org/wiki/Haptic (2008-03-02) [5] Homepage of SenseGraphics. http://www.sensegraphics.com (2008-05-05) [6] Homepage of SensAble. http://www.sensable.com (2008-05-05) [7] Homepage of Novint. http://home.novint.com/ (2008-05-05) [8] Haptic Simulation Surgery Project. Mechatronics Lab,KTH.

http://www.md.kth.se/research/projects/mda/p13.shtml?eng (2008-05-05) [9] Weber, Katz and Beyond; An Introduction to Psychological Studies of Touch and

the Implications for and Understanding of Astists' Making and Thinking Processes. “So what is Haptics anyway?” http://www.biad.uce.ac.uk/research/rti/riadm/issue2/abstract.htm (2008-03-02)

[10] Takuya Nojima, University of Tokyo; Dairoku Sekiguchi, University of Tokyo;

Masahiko Inami, University of Tokyo; Susumu Tachi, University of Tokyo; The SmartTool: A system for Augmented Reality of Haptics http://csdl2.computer.org/persagen/DLAbsToc.jsp?resourcePath=/dl/proceedings/&toc=comp/proceedings/vr/2002/1492/00/1492toc.xml&DOI=10.1109/VR.2002.996506 (2008-02-26)

[11] Immersion Corporation, Value of Haptics,

http://www.immersion.com/corporate/press_room/pdfs/Value-of-Haptics_May07v1-lr.pdf (2008-02-27)

References ________________________________________________________________

62

[12] Free online dictionary of English synonyms, http://www.sensagent.com/alexandria-online/getDataAlexandriaOnline.jsp?w=simulation&sl=en&tl1=en&dl=en#synonyms (2008-02-28)

[13] F. Permadi, Ray-Casting Tutorial, http://www.permadi.com/tutorial/raycast/

(2008-03-07) [14] Karon MacLean, "Haptics and the User Interface", based on slides, original slides

available at: http://www.cs.ubc.ca/~maclean/publics/ (2008-03-02) [15] Six degrees of freedom, http://en.wikipedia.org/wiki/6DOF (2008-03-10) [16] OpenHaptics, http://www.sensable.com/products-openhaptics-toolkit.htm (2008-

03-02) [17] Real-Time collision detection engine, SmartCollision,

http://haptics.net/products/Haptic_Gallery/SmartCollision.asp (2008-03-05) [18] The Marching Cubes Algorithm, http://www.exaflop.org/docs/marchcubes/

(2008-05-12) [19] Teledildonics. http://en.wikipedia.org/wiki/Teledildonics (2008-02-25) [20] Teledildonics. http://www.teledildonics.com/ (2008-02-25) [21] Data sheet, PHANTOM® Omni™,

http://www.sensable.com/documents/documents/PHANTOM_Omni_Spec.pdf (2008-03-23)

[22] Data sheet, PHANTOM® Premium 1.5/6DOF,

http://www.sensable.com/documents/documents/Premium_1.5_6DOF.pdf (2008-05-05)

Appendix ________________________________________________________________

63

Appendix The pseudocode for the created algorithms is presented below.

Overview of “The Line Algorithm” //calculate the haptic force feedback: if( Tip of the tool ( ProbePoint) has collision) { Find ProxyPoint

Calculate haptic force feedback, F } else //Tip of the tool (ProbePoint) has NO collision { Set ProxyPoint = ProbePoint

Set F = (0,0,0) } Move out ProxyPointOffset

//calculate the haptic torque feedback: Put vertices along the centreline of the cylinder

Collision detection of these vertices at time step t

Check if translated or rotated (moved) the tool from t-1 to t

if( moved the tool) { if(NO collision at t)

{ NO_collision = true

} else //collision at t { if(collision at t-1) { Find IntersectionPoint based on earlier developed algorithm } else //NO collision at t-1 { Find IntersectionPoint based on Translation/Rotation

method or ray casting method } } } else // NOT moved the tool { if(collision at t-1) { IntersectionPoint at t = IntersectionPoint at t-1 } else //NO collision at t-1

Appendix ________________________________________________________________

64

{ NO_collision = true } } if(NO_collision) { Set the haptic torque feedback, Torque = (0,0,0) } else //collision { Calculate ForceReaction at the IntersectionPoint and the Torque } Move out ProxyCylinderOffset

Overview of “The Surface Algorithm” //calculate the haptic force feedback: if(Tip of the tool (center point of the sphere) has NO collision) { for every vertex i on the surface of the sphere { Calculate the position for vertex i and put it there

Collision detection of vertex i if(Collision of vertex i) { Calculate the force from vertex i

Add this force to the total force for the sphere } else //NO collision of vertex i { The total force for the sphere remains the same } } } else //Tip of the tool has collision { The total force for the sphere at time step t is the same as in t-1 } //calculate the haptic torque feedback: for every slice j { if(Center point of the slice has NO collision) { for every vertex i on the radius of the slice { Calculate the position for vertex i and put it there

Collision detection of vertex i if(Collision of vertex i) { Calculate the force from vertex i

Add this force to the reaction force for the slice

Appendix ________________________________________________________________

65

} else //NO collision of vertex i { The reaction force for the slice remains the same } } } else //Center point of the slice has collision { The reaction force for the slice at time step t is the same as in t-1 } Calculate the vector r from the tip of the tool to the center point of slice j

Calculate the torque for slice j as vector r cross product the reaction force for slice j

Add this torque to the total torque }

TRITA-CSC-E 2008:068 ISRN-KTH/CSC/E--08/068--SE

ISSN-1653-5715

www.kth.se

Documents

Multipoint Collision Detection Algorithm for Bone Milling and