Upload
mohammed-a-isa
View
227
Download
0
Embed Size (px)
Citation preview
8/10/2019 Tool Monitoring Using Image Processing
1/21
http://dx.doi.org/10.1016/j.cirpj.2013.02.005http://www.sciencedirect.com/science/journal/17555817mailto:[email protected]://dx.doi.org/10.1016/j.cirpj.2013.02.005http://crossmark.dyndns.org/dialog/?doi=10.1016/j.cirpj.2013.02.005&domain=pdfhttp://crossmark.dyndns.org/dialog/?doi=10.1016/j.cirpj.2013.02.005&domain=pdf8/10/2019 Tool Monitoring Using Image Processing
2/21
from the results of signal processing techniques. Then prediction of
process data and process optimization canbe possible usingdesign
of experiment (DoE) and artificial intelligence (AI) techniques from
the extracted and selected features. Comparison of actual and
predicted values of selected features are also required to find out
the precision of that technique. Then optimized data are fed to the
machine controller and servo mechanism which can control the
machining process. Elbestawi et al. [34] comprehensively classified
different sensor systems for monitoring different output process
parameters viz. dimensions, cutting force, feed force, spindle
motor and acoustic emissions used in turning, milling and drilling
operations. Two excellent case studies have been conducted by
them using proposed multiple principal component fuzzy neural
network for classification of sharp tool, slightly worn tool, medium
worn tool, severe worn tool and breakage in turning and drilling
experiment using force, vibration and power signal. An online
monitoring of chipping in drilling process has also been conducted
by them using vibration signal with 97% success rate. Roth et al.
[106] emphasized wireless, integrated and embedded low cost
sensors; wavelet, time-frequency and time-scale analysis as a
signal processing approach; artificial neural network (ANN) and
support vector machine approach for assessment of tool condition;
hidden Markov model and recurrent neural network for the
prediction purpose in their comprehensive review of TCM forturning, milling, drilling and grinding processes. Nebot and
Subiron [92] reviewed the TCM systems of machining and
proposed a generic methodology combining DoE and ANN for
improved process modelling and prediction. Teti et al. [121] made
a comprehensive review on intelligent sensors for monitoring and
control of advancedmachining operation. They also mentioned the
real industrial implementationof the intelligent sensor systems for
TCM of advanced machining of complex-shaped parts made of
super alloy. Chandrasekaran et al. [19] made an comprehensive
literature review on the application of soft computing techniques
viz. neural network, fuzzy logic, genetic algorithm, simulated
annealing, ant colony optimization and particle swarm optimiza-
tion on turning, milling, grinding and drilling operations for
optimization of cutting conditions with minimum cost machiningwith maximum production rate based on prediction of process
outputs viz. surface finish, cutting force and tool wear.
The product quality is principally dependent on the machined
surface. The surfacequality ismainlydependent on the cutting tool
wear. Cutting tool wear is dependent upon cutting conditions,
work and tool material, tool geometry. There are four modes of
cutting tool wears, such as, adhesive wear due to shear plane
deformation, abrasive wear due to hard particles cutting, diffusion
wear due to high temperature and fracture wear due to fatigue.
Four principal types of wear occur in cutting tool and they are nose
wear, flank wear, crater wear and notch wear. Flank wear (as
shown in Fig. 1) occurs due to rubbing between tool flank surface
and work piece. Flank wear is specified by maximum flank wear
width (VBmax) or mean flank wear width (VBmean). Tool life
criterion is mainly dependent on the VBmean. Cutting tools are
experiencing three stages of wear [29] viz. initial wear (during first
few minutes), steady-state (cutting tool quality slowly deterio-
rates) and severe wear (rapid deterioration as the tool reaches the
end of its life). Crater wear are produced at the due the high
temperature for chip-tool interaction. This wear is characterized
by the crater depth and crater area.
Principally, tool condition monitoring systems can be classified
into two groups. They are, (a) direct techniques and (b) indirect
techniques. In direct techniques, flank wear width, crater depth
and crater area are measured directly either with tool makers
microscope, 3D surface profiler, optical microscope or scanning
electron microscope (off-line method) or with CCD camera (in-
process method). In indirect techniques, the measured parameters
or signals (viz. force, acoustic emission, current, power, surface
finish, etc.) of the cutting process allow for drawing conclusions
upon the degree of tool wear. Normally, these tool wear
monitoring systems are based upon the comparison of a reference
signal of an optimized cutting process with the actual process
signal [127]. These techniques have predominantly been imple-
mented, employing such varied technologies as acoustic emission,
cutting force, spindle current, and vibration sensors [99]. However,
there are some limitations of these methods. To overcome thoselimitations, research is going on to identify the degree of tool wear
by analyzing surface texture of machined surfaces with digital
imageprocessing technique from the images ofmachined surfaces.
There is a wide range of application of digital image processing
(DIP) using machine vision in machining processes like control of
surface quality, tool wear measurements, work piece surface
texture measurements, etc.
1.1. Advantages and disadvantages of DIP for tool condition
monitoring
There are some advantages of using digital image processing
techniques over other techniques to monitor any manufacturing
process.Such as, (1) it appliesno forceor load to the surface textureunder examination; (2) it is a non-contact, in-process application
[63]; (3) this monitoring system is more flexible and inexpensive
than other systems; (4) this system can be operated and controlled
from a remote location, so it is very much helpful for unmanned
production system; (5) this technique is not dependent on the
frequency of the chatter, directionality as acoustic emission (AE)
sensors are dependent on those factors; also, the AE sensors are
mainly detecting tool breakage inmachining [17,102,29]. Thus, the
monitoring of progressive wears of cutting tool is very difficult
using AE sensors; (6) vibration sensors (accelerometer) can
monitor tool breakage, out of tolerance parts and machine
collisions [52]; the progressive wear monitoring has not been
possible using vibration sensors; (7) DIP technique is not affected
Fig. 1. Flank wear and notch wear from the microscopic image of a tool insert.
S. Dutta et al./ CIRP Journal of Manufacturing Science and Technology 6 (2013) 212232 213
8/10/2019 Tool Monitoring Using Image Processing
3/21
by the high frequency forces as this high frequency forces cannot
be taken by dynamometer; also the force sensors are sensitive to
machine vibrations [53]; (8) to monitor and control a machining
process, the fusion of several sensors (AE sensor, dynamometer,
vibration signatures, etc.) is required, which is not at all cost
effective [52]; (9) however, the machined surface image carries the
information of tool imprint as well as the change of tool geometry
[9]; thus, a roughness, waviness and form information can be
obtained by analyzing a machined surface image [15]; (10) a 2D
information canbe obtained from a machined surface image which
is not possible to get by a 1D surface profiler [122]; (11) also, the
information of machining parameters can be obtained from
machined surface images [31]; (12) the development of CCD
cameras has also contributed to the acceptance of industrial image
processing, since CCD cameras are less sensitive to the adverse
industrial environment; (13) optical image processing has brought
about the possibility of adding, subtracting, multiplying, storing
and even performing different image transforms using optical
devices; (14) three dimensional surface roughness of machined
surface can be measured, accurately, using scanning type 3D
surface profiler [1,23,88,95]; however, these 3D measurements are
not effective for in-process or online tool conditionmonitoring due
to uneconomic time, cost ineffectiveness and inaccessibility to the
machine tools; to overcome this situation, a machine vision basedsystem can be useful for monitoring purpose. However, there are
some limitations for using machine vision system in tool condition
monitoring techniques also [141]. (1) An appropriate illumination
system, robust image processing algorithm, protection from
machining noises (chips, dirts, etc.) are very much essential for
the successful implementation of this technique [9]. (2) Monitor-
ingofdrillpartsusing DIP are verydifficult due to its inaccessibility
[51]. However, a method to monitor deep hole parts has been
developed in recent years [84].
This paper is composed of five major components. The first
component presents an overview of digital image processing
techniques used for tool condition monitoring. The second
explains lighting systems which are used in TCM. The third
presents direct TCM techniques usingdigital imageprocessing.Thefourth component presents different in-direct TCM techniques
using image processing. And the final and last component draws
overall conclusions and suggests future directions for TCM
research through digital image processing technique.
2.
Digital
image
processing
techniques
Image acquisition is the first step of any machine vision system.
In case of TCM, images of cutting tool (rake face or flank surface) or
work piece surface are captured with a CCD (Charged Coupled
Device) camera or CMOS (Complementary Metal-Oxide Semicon-
ductor) digital camera. CCD camera is comprised of CCD sensor
which
is
an
array
of
photosensitive
elements
to
collect
electricalcharges generated by absorbed photons. Those electrical charges
are then converted to an electrical signal which is converted to a
digital image via frame grabber. Finally, the image is transferred to
a PC for processing purpose [50]. CMOS is different from CCD
sensor by its faster capturing rate. CMOS sensor can acquire frames
faster than CCD camera. But the sensitivity of CMOS sensor ismuch
less than CCD sensor. To create a digital image, a conversion is
needed from the continuous sensed data into digital form. This
involves two processes: sampling and quantization. Digitization of
coordinate values and amplitude values are called sampling and
quantization. Image magnification is also possible by linear
interpolation, cubic interpolation, cubic convolution interpolation
etc. Different types of neighbourhood operations are also needed
for
further
processing
[41].
From the illumination point of view, an Image f(x, y) may be
characterized by two components: (1) the amount of source
illumination incident on the scene, and (2) the amount of
illumination reflected by the objects. Appropriately, these are
called the illumination and reflectance components and are
denoted by i(x, y) and r(x, y), respectively. The two functions
combine as a product to form f(x, y),
f x;y ix;yrx;y (1)
Image pre-processing is required for the improvement of
images by contrast stretching, histogram equalization, noise
reduction by filtering, inhomogeneous illumination compensa-
tion etc. To increase contrast in an image, contrast stretching and
histogram equalization are two mostly used techniques. To
reduce noise, low pass filtering is very important technique. It
includes image smoothing by using low pass filtering in both
spatial and frequency domains. In spatial low pass filtering, a
filter mask is convolved with the image matrix to reduce
unwanted noise present in the image (image smoothing). Order
statistics or median filter is used to remove impulse noise in an
image (image smoothing). Butterworth and Gaussian low pass
filters are some common low pass filters in frequency domain.
High pass filters are used to enhance the sharpness of an image(image sharpening). Unsharp masking (to emphasize high
frequency components with retaining low frequency compo-
nents), Laplacian filter (second order filter) are some spatial high
pass filters used for image sharpening purpose [41]. Image
filtering and enhancement operations are very much essential to
reduce the noise of the images specially for cutting tool images,
because there are a chance of noise due to the dirt, oils, dust of
machining on the object surface. The low-pass filtering (e.g.
median filter, Gaussian filter, etc.) is useful to reduce the noises
present in the cutting tool wear images and machined surface
images. Also the high pass filtering technique can be useful to
enhance tool wear profile and for clear identifications of feed
marks in machined surface images.
After pre-processing, image segmentation and edge detectionare generally done to segment the worn region of cutting tool
from the unworn region and also to detect the edges of the feed
lines of themachined surface images. Image segmentation is the
method of partitioning an image intomultiple regions according
to a given criterion. Feature-state based techniques collect pixel/
region properties into feature vectors and then use such vectors
for assigning them to classes, by choosing some threshold
values. While feature-state based techniques do not take into
account spatial relationships among pixels, image-domain based
techniques do take them into account; for example, split and
merge techniques divide and merge adjacent regions according
to similarity measurements; region growing techniques aggre-
gate adjacent pixels starting fromrandomseeds (region centres),
again by comparing
pixel values. Watershed-based segmenta-tion technique can be useful for micro and nano surface
topography. Watershed analysis, which consists in reasoning
over a surface topography in terms of hills and dales, actually
originates from the work by Maxwell on geographical analysis.
Watershed-based surface segmentation consists in partitioning
the surface topography into regions classified as hills (areas from
which maximum uphill paths lead to one particular peak) or
dales (areas from which maximum downhill paths lead to one
particular pit), the boundaries between hills being watercourse
lines, and the boundaries between dales being watershed lines
[2].
The edge detection operation is used to detect significant edges
of an image by calculating image gradient and direction. Gradient
and direction of an image f(x, y) are defined in Eqs. (2) and (3),
S. Dutta et al./CIRP Journal of Manufacturing Science and Technology 6 (2013) 212232214
8/10/2019 Tool Monitoring Using Image Processing
4/21
respectively.
GxGy
d f
dxd f
dy
2664
3775 (2)
ux;y tan1 GyGx
(3)
where u is measured with respect to the x-axis.
Robert operator (sensitive to noise), Sobel operator, Prewitt
edge operator are some first order edge detectors which are very
useful for automatic detection of tool wear profile. Canny edge
detector is widely used in the field of machine vision because of its
noise immunity and capability to detect true edge points with
minimum error. In Canny edge detection method, the image is first
convolved with Gaussian smoothing filter with standard deviation
s. This operation is followed by gradient computation on the
resultant smoothed image. Non-maxima suppression, double
thresholding and edge threshold selection with Bayes decision
theory are the steps to implement Canny edge detection. Gradient
images of tool flank wear (experimentally obtained from milling
operation) and machined surface (experimentally obtained from
turning operation) usingCanny edge detector are shown in Fig.2. Awear profile or edges of surface texture can be obtained by this
method. The edge detector based on double derivative is used to
detect only those points as edge points which possess local
maxima in the gradient values. Laplacian and LaplacianofGaussian
are the most commonly used double derivative-based edge
detectors.
For partitioning a digital image into multiple regions, grey level
thresholding techniques are computationally inexpensive. Based
on some optimal threshold, an image can be partitioned into
multiple regions. For example, to partition the flank wear profile
from itsbackground, thresholding techniques are generallyused. A
very common thresholding technique used in tool wear measure-
ment is Otsus optimal thresholding technique. In this technique, a
class, C0 is formed with all the grey value V(k) for a grey levelintensity, k and all the other form another class, C1. Optimal k value
is selected for maximum between-class variance. In bi-level
thresholding technique imagesarepartitioned into foreground and
background segments and in multilevel or dynamic thresholding,
images are divided into more than two segments. In entropy-based
thresholding, the threshold value is selected in such a way, so that
the total entropy value of foreground and background is maximum
[2]. Thresholding techniques are important forbinarizationofflank
wear profile.
After edge detection and thresholding, morphological opera-
tions viz. erosion, dilation, closing, opening are important tools for
completing the wear profile, accurately. In this operation, a
noiseless morphology is obtained by introducing or removing
some
points
or
grey
values
in
a
profile
[41].Tool condition monitoring via surface texture of machined
parts are mainly dependent on the texture analysismethod. This
methodcan be applied after pre-processing. Texture is a repeated
pattern, whichisa setof local statisticsorattributes vary slowlyor
remain approximately periodic. Primitive in texture is a con-
nected set of pixels, characterized by a set of attributes
(coarseness and directionality). For example, in case of turned
surface, a repetitive feed marks can be obtained as texture
primitives. Texture analysis can be done using statistical,
geometrical, model-based and signal processing basedmethods.
In statistical method a texture is modelled as a randomfield and a
statistical probability density function model is fitted to the
spatial distribution of intensities in the texture. Higher-order
statistics like run-length statistics,
second order statistics like
grey level co-occurrence matrix (GLCM) can beused as statistical
texture classifiers. In geometric texture analysis method, the
analysis depends upon the geometric properties of texture
primitives. Voronoi tessellation, Zuckers model are some of the
geometric texture analysis methods. In model based methods,
texture analysis is done with some signal model like, Markov
random field, Gibbs random field, Derin-Elliot, auto-binomial,
fractal (self-similarity) models are some mathematical model-
based texture analysis methods. In signal-processing based
texture analysis, spatial domain filtering, Fourier-domain filter-
ing, Gabor and wavelet analysis are some common texture
analysis methods [125].
3.
Lighting
systems
Lighting system is the most important and critical aspect to
receive a proper image for image processing. Due to inhomoge-
neous illumination for improper lighting set-up, the information
from images will not be sufficient for any machine vision
application. Several researches give strongimportanceon lighting
set-up for tool condition monitoring using image processing.
Lighting systems required are varying depending on applications
viz. for capturing tool wear image and machined surface image.
Weis [132] triedto capture thetoolwear imageusing adiodeflashlight incorporated with a infrared band filter, which helped to
enhance the tool wear region with respect to the background.
KuradaandBradley [73] usedtwofibre-opticguides tocapture the
tool wear regions. They used it to obtain adequate contrast
between the worn and unworn tool regions. Pfeifer andWeigers
[99] usedring of LEDs attachedwith camera to capturethe proper
illuminated images of tool inserts from different angle. Kim et al.
[70] useda fibre optic light surrounding the lens to illuminate the
flank face portion of a 4-fluted end mill. They also examined that
the best measurement of flank wear can be possible with a high
power lighting (60 W). Jurkovic et al. [58] utilized a halogen light
to illuminate the rake andflank face of the cutting tool and a laser
diode and accessories to obtain a structured light pattern on the
face of the tool to detect the tool wear by the deformation ofstructured light on the rake face. Wang et al. [131] used a fibre
optic guided light to illuminate the flank portion of each insert
attached to a 4-fluted milling tool holder and capture the
successive images in a slow rotating condition by using a laser
trigger with very less blurring. A white light from a fluorescent
ring as well as light from a fibre bundle was used to minimize
specular reflections on capture the tool imagesby Kerr et al. [68].
So, highly illuminated and directional lighting is required to
capture the tool wear region as to get a very accurately
illuminated image. Wong et al. [134] used a 5 mW HeNe laser
0.8mm-diameter beamfor focusingonto themachined surfaceby
a lens at an incident angle of 308 for capturing the centre of the
pattern. Then the reflected light pattern was formed on a screen
made
of
whitecoated glass from
where
the
scattered patternwasgrabbed using a CCD camera. The setup was covered in order to
minimize interference from ambient light and a consistent
lighting condition for all the tests has been provided. But the
actual image of the machined surface is required instead of
reflectedpattern.Tsai etal. [123],triedtoobtaina homogeneously
illuminatedmachined surface image bya regularfluorescent light
source which was situated at an angle of approximately 108
incidence with respect to the normalof the specimen surface. The
camera was also set up at an angle of approximately 108 with
respect to the normal of the specimen surface to obtain image at
the direction of light. But this set-up may only be useful for flat
specimens not for curved surfaces. Bradley and Wong [16] used a
fibre optic guided illumination source and a lighting fixture. A
uniform
illumination of
the
machined surface was
ensured
by
S. Dutta et al./ CIRP Journal of Manufacturing Science and Technology 6 (2013) 212232 215
8/10/2019 Tool Monitoring Using Image Processing
5/21
changing the position of lighting fixture. During surface assess-
ment, the specimen was positioned on the platformso thatthe lay
marks were perpendicular to the longer dimension of the CCD
sensor. The light sourcewas positioned at a distance of 8 cm from
the surface, as
this
provided
the
best image
contrast. In
thistechnique, the images of flat specimens (end milled) were
captured but the images of turned surface (i.e. curved surfaces)
were not obtained. Leeet al. [78] useda diffused,blue light source
situated at an angle of approximately 458 incidence with respect
to the machined (turned) surface specimen to accomplish the
illumination of the specimens. Alegre et al. [4], explained about a
diffusedlighting system(aDC regulated light sourcewith infrared
interference filter for cool illumination) for capturing images of
turned parts. They also used a square continuous diffused
illuminator for getting diffused illumination in the camera axis.
The last lighting system is most appropriate for obtaining a
homogeneously illuminated image of turned or curved parts. A
cover can beused to reducethe interference of ambient lighting in
industrial environment.
4. Direct TCM techniques using image processing
There are two predominant wear mechanisms for a cutting
tools useful life: flank wear and crater wear. Flank wear occurs
on
the relief
face of
the tool and is
mainly attributed
to therubbing action of the tool on the machined surface. Crater wear
occurs on the rake face of the tool and changes the chip-tool
interface, thus affecting the cutting process. Tool wears increases
progressively during machining. It depends on the type of tool
material, cutting conditions and lubricant selected. Online
measurement of tool wear by image processing after taking
images of cutting tool through machine vision system is under
research. This technique is coming under the area of direct tool
condition monitoring. Flank wear can directly be determined by
capturing images of cutting tool but a more complex technique is
required to determine the crater depth [59]. Cutting tool wears
have been measured by two dimensional and three dimensional
techniques in various researches which are described in the
following
sections.
Fig. 2. (a) Milling tool wear image and (b) corresponding gradient image using Canny edge detector (c) turned surface image and (d) corresponding gradient image using
Canny edge detector.
S. Dutta et al./CIRP Journal of Manufacturing Science and Technology 6 (2013) 212232216
8/10/2019 Tool Monitoring Using Image Processing
6/21
8/10/2019 Tool Monitoring Using Image Processing
7/21
8/10/2019 Tool Monitoring Using Image Processing
8/21
function for accurate segmentation of wear contour of cutting
inserts used for milling operation. They measured the tool wear
area by this method. However, the measurement of flank wear
width had been missing in their research.
Otieno et al. [96], studied flank wears of two fluted micro end
mills ofdiameter 1 mm,0.625 mm and0.25 mmwithdigital image
processing techniques using filtering and thresholding by XOR
operator. But any edge detection, tool wear quantification and
wear classification was not performed.
Inoue et al. [47] made a generalized approach by detecting
defects in rod-shaped cutting tool via edge detection (by Prewitt
operator) and extracted image parameters after performing
discrete Fourier transform (DFT) on the edge detected image.
However, many unstudied defects cannot be possible to recog-
nized by this system.
Jackson et al. [49] utilized aneural imageprocessingmethod for
accurate detection of very small wear developed in very small
diameter milling cutter on the environmental scanning electron
microscopic (ESEM) images of tool. They have even measured the
small average wear of 5mm developed in a 9.5 mm diameter
milling cutter. Though this technique is very much useful for
micro-machining, but the method is very much difficult to use
online.
Grain fracture, bond fracture and attritous wear are three typesofpre-dominant wears in grindingwheel.Wear flats aredeveloped
on the grinding wheel surface due to attritous wear. Consequently,
the increasing rate of wear flats area develops heat and burn the
workpiece. But the automatic and precise segmentation of true
wear flats are quite challenging task from the wheel surface
images. An edge detection approach after thresholding were
utilized to distinguish true wear flats from its background [138].
However, the accurate selection of intensity threshold and edge
threshold was a difficult task. To overcome this problem, Lachance
et al. [75]utilizeda region growingmethod for segmenting the true
wear flats from its background. However, some morphological
techniques can be utilized for more accurate computation of wear
flat area. Heger and Pandit [43] captured the images of grinding
wheel surface by multidirectional illumination and image fusionfor obtaining more detailed information. Then they have utilized
multi-scale wavelet transform and classification technique for
distinguishing the grains and cavities on the surface. A new
approach to discriminate the fresh and worn out grinding wheels,
progressively, has been established by Arunachalam and Rama-
moorthy [10]. They extracted some texture descriptors for
describing the condition of grinding wheel surface utilizing
histogram based, GLCM based and fractal based texture analysis
methods on the wheel surface images taken at different progres-
sive time. However, no explanation regarding the variations of
selected features with the progressive wear has been encountered.
In the area of integrated circuit (IC) manufacturing, the surface
of stamped tool or cutting dust has been monitored real time by
Kashiwagi
et
al.
[62].
They
captured
the
surface
image
of
cuttingdust and determined the width of stamped line by using image
histogram and cross-correlation technique. They observed that the
width was decreasingwith the increase of cutting time or decrease
of tool sharpness.
4.2.
Three
dimensional
techniques
Three dimensional measurement techniques are used to
measure the crater depth accurately. Yang and Kwon [137,136]
first used a microscope equipped with a CCD sensors to capture
noisy images of rake face of an worn out tool insert and measured
thedepth of crater indifferent levels ofwearby automatic focusing
technique. They have used image consolidation and median
filtering
to
remove
high
frequency
noises
without
blurring
from
rake face image. Then they thresholded optimally for segmenting
the worn region from the background and detected the crater
contour by using Laplacian method. Edge linking and dilation
methods incorporating eight neighbourhood chain coding have
been applied on that contour to get an accurate shape of crater
region. A Laplacian criterion function incorporating an infinite
impulse response (IIR) filter has been used for getting the focused
position along z-direction. A hybrid search algorithm with
polynomial interpolation and golden search technique has been
utilized to improve the accuracy of the automated focusing
technique, in this method. This way they measured the crater
depth. They used seven features (four were related to flank wear
and three were related to crater wear) to classify flank wear, crater
wear, chipping and fracture. A mathematical model was intro-
duced in their work to obtain flank wear profile from crater wear
contour. Then they selected 12 input nodes (each node contains
seven feature parameters) and 4 output nodes (flank wear, crater
wear, chipping and fracture) in a multi-layer perceptron (MLP)
neural network to classify four types of wear. All the tests were
done on a P20 cemented carbide tool insert without chip breaker.
Though the work is pioneering the crater depth measurement very
accurately, but the 3D map of crater region has not been evaluated
by this offline technique. Also, it may be difficult to use their
technique for insert with chip breaker due to the major undulationof rake surface. Ramamoorthy and co-workers [61,100]used image
processing with stereo vision technique with only a single CCD
camera todetermine thedepth of each point in the crater. Trends of
tool wear pattern were then analyzed with a MLPNN algorithm,
where inputs were speed, feed, depth of cut and cutting time and
output parameters were flank wear width and crater wear depth.
However, the crater depth estimation less than 125 mm could not
be obtained accurately by this technique. Also some pre-
processing algorithm were required to eliminated the noises from
dirt, chip, oil etc. on the rake face to make the method possible in
online.
Ng and Moon [93] proposed a technique for 3D measurement of
tool wear for micro milling tool (50 mm diameter) by capturing
images with varying the tool and camera plane distance with15 mm resolution. Then they have re-constructed 3D image from
the captured imagesusing digital focusmeasurement. Finally, they
proposed that the tool wear measurement could be possible by
combining the actual 3D image and the 3D CAD model of the tool.
However, no depth measurement had been performed in their
work.
Devillez et al. [24] utilized white light interferometry technique
to measure the depth of crater wear and determined the optimal
cutting conditions (cutting speed and feed rate) to get the best
surface finish in orthogonal dry turning of 42CrMo4 steel with a
uncoated carbide insert. In white light interferometry technique, a
vertical scanning has been performed to get the best focus
positions for each and every point presented in the object to be
measured.
White
light
is
used
to
get
the
high
resolution
(sub-nanometer) and high precision measurements over a wider area.
However, this technique is an offline technique and the measure-
ment of crater depth of grooved inserts or inserts with chip breaker
is quite challenging for this technique. Dawson and Kurfess [22]
used a computational metrology technique to determine the flank
wear and crater wear rate of a coated and uncoated cubic boron
nitride (CBN) tool for progressive wear monitoring in offline. They
have acquired the data of the worn out cutting insert by using
white light interferometry and compute the volume reduction in
the insert by comparing those data with the CAD model of fresh
insert developed by using computational metrology. However, no
grooved insert has been used in their technique. Wang et al. [128
131] measured various parameters viz. crater depth, crater width,
crater
centre
and
crater
front
distance
of
crater
wear
by
S. Dutta et al./ CIRP Journal of Manufacturing Science and Technology 6 (2013) 212232 219
8/10/2019 Tool Monitoring Using Image Processing
9/21
Table 1
Direct TCM techniques based on image processing.
Researcher Illumination sys-
tem
Image processing Type of tool wear
measurement
Machining Remarks
Galante et al. [40] Diffused lighting Thresholding Flank wear Turning Offline, 2D technique
Weis [132] Diode flash light
with
infra-red
band
filter
Dilation and thresholding Flank wear
measurement
Milling No evaluation of accuracy
Kurada and
Bradley
[73]
Fibre optic guided
light
Image enhancement, Image
segmentation,
thresholding,morphological operation
Flank wear
measurement
Turning Offline
Tauno and
Lembit [120]
Blue light source Median filter, Roberts edge
detector, thresholding
Flank wear
measurement
Turning, milling Offline, 8% error
Pfeifer and
Weigers [99]
Ring of LED Method to set optimum
incidence angle of lighting for
controlled illumination
Flank wear
measurement
Turning, milling Online
Sortino
[116]
Median
filtering
Statistical filter for edge
detection
Flank
wear
measurement
Generalized
for insert
Offline
Flank wear
Jurkovic
et
al.
[58]
Halogen
light
along
with a laser diode
Manual
measurement
using
a
image processing software
Flank
wear
and
deformation of
laser light pattern
on rake face
Tool
inserts
Manual
measurement,
crater
depth measurement has not
been done
Wang et al.
[128131]
Laser trigger
synchronized with
camera, fibre optic
guided light
Find critical area, find reference
line, pixel to pixel scan for
measuring VBmaxfrom reference
line
Flank wear
(captured when
tool is moving)
Milling inserts Online, max error 15mm,
difficult to measure coated
carbide inserts
Liang et al. [83] Backlighting Image registration, spatial
transformation, image
subtraction,
similarity
analysis
Nose wear Inserts Difficult to implement for
flank wear width
measurement
Sahabi and
Ratnam [108]
Backlighting Weiner filter, thresholding,
detection and subtraction of
wornandunwornprofile in polar
co-ordinate
Flank wear from
nose radius and
surface roughness
profile
Inserts 7.7% (from nose) and 5.5%
error (from surface
roughness), difficult to
implement in very low feed
application
Fadare and Oni [35] 2 incandescent
light sources
inclined at 458
Weiner filter, shadow removing,
canny edge detection, pixel
counting
Flank wear Inserts Sensitive to the fluctuation of
ambient light
Kerr et al. [68] White ring light,
fibre
optic
guided
light
Unsharp mask, manual
measurement,
histogram
analysis, GLCM analysis, Fourier
spectrum analysis, fractal
analysis
Flank wear
measurement
via
texture descriptors
Turning inserts,
end
mill
cutter
Texture analysis of wear
region,
no
automatic
measurement of wear
Lanzetta [76] Structured lighting
with Laser
Resolution enhancement,
averaging, segmentation
Flank and crater
wear
Generalized
for insert
The effect of dirt, oils on
inserts did not addressSchmitt et al. [112] and
Stemmer et al. [117]
Ring light (for full
and side
illumination)
Sobel filter, line interpolation,
histogram transformation,
morphological opening &
closing, blob analysis, contour
detection
for
measurement;
NN
for flank wear and breakage
classification
Flank wear
measurement,
wear and breakage
classification
Milling Resolution 4.4mm,
classification error 4%; the
method has not been applied
for different variety of cutting
inserts
Castejon et
al.
[18]
and
Barriero et al. [13]
DC
regulated
light
with square
continuous
diffused
illuminator
Low
pass
filter,
cropping,
histogram stretching, manual
segmentation,moment invariant
methods (Zernike, Legendre, Hu,
Taubin, Flusser), and linear
discriminant analysis for
classification
Classification
of
low, medium and
high wear
Inserts
99.88%
discrimination
for
Hus descriptor, no wear
prediction has been
performed
Alegre et al. [6] DC regulated light
with square
continuousdiffused
illuminator
Contour signature based on
Canny edge detected image, k-
NN
and
MLPNN
for
classification
Classification of
low and high wear
Inserts 5.1% classification error;
three levels of wear
classification
is
needed
Atli et al. [11] Silhouette image of
tool
Canny edge detection,
measurement
of
deviation
from
linearity of tool tip
Drill-bit Drilling Only useful for drilling; Flank
wear
width
cannot
be
measured
Makki et al. [86] Silhouette image of
tool captured at
1001500 r.p.m
Canny edge detection, best
fitting algorithm
Tool run out
detection
Drilling Tool run-outperpendicular to
the image plane had not been
measured
Liang and Chiou [82] Circular back
lighting
Spatial moment edge detection,
edge sorting, B-spline
smoothing,
gaussian
LPF,
thresholding, morphological
operation
Flank wear
detection for
progressive
machining
Multi-layer
twist drill
Results were not compared
with the microscopic wear
measurement;
applicable
for
no smear image
Su
et
al.
[118]
Circular
lighting
Accurate
edge
detection
proposed, rotation, automated
measurement
Flank
wear
detection for
progressive
machining
Micro
drill-bit
(for PCB drilling)
Resolution
0.996
mm/pixel;
only applicable when no
smearing in cutting plane
image
S. Dutta et al./CIRP Journal of Manufacturing Science and Technology 6 (2013) 212232220
8/10/2019 Tool Monitoring Using Image Processing
10/21
reconstructing a 3D crater profile by capturing four fringe patterns
with four phase shifting angles. No scanning is required in this
method unlike white interferometry technique. However, the
accuracy of the measurement is dependent on the fringe width or
fringe pattern.
Table 1 summarizes the application of digital image processing
in direct tool wear monitoring.
So, in direct technique, condition monitoring is done by
analyzing the change in geometry of the cutting tool. Chatter,
vibration, cutting force change etc. are not taken into account with
cutting tool observation whereas surface finish can emphasize
those changes as well as change in tool geometry. So, researchers
are
going
to
take
the
measurement
of
surface
finish
throughindirect TCM techniques using image processing of machined
surface images.
5. Indirect TCM techniques using image processing
Diversepropertiesplay an important role in the surfacefinish of
metallic parts, e.g. mechanical strength, wear resistance of the
surfaces or geometrical and dimensional quality of the parts. These
properties are directly related to the surface finish level, which is
dependent on the manufacturing process parameters and the
materials used. Thus, the measurement of the surface finish has
been a research matter of special interest during last sixty years in
machining sector. There are tactile and non-tactile techniques to
assess
the
surface
quality
of
the
machined
parts.
In
tactile
techniques, surface roughness parameters are measured using a
stylus instrument; whereas in non-tactile method, surface
roughness parameters are obtained from the images of machined
surface textures.But there is a chance of scratches on softmaterials
in tactile techniques due to the tracking of stylus on measurable
surface; whereas non-tactile techniques are becoming more
advantageous due to the advancement of computer vision
technology. While tactile techniques characterize a linear track
over the surface of the part, the computer vision techniques allow
characterizing whole areas of the surface of the part, providing
more information [8,111,113]. Besides, computer vision techni-
ques take measures faster, as images are captured in almost no
time
and
so
they
can
be
implemented
in
the
machine.
According
tothis, it is possible to apply these techniques for controlling the
processes in real time on an autonomous manner. An exhaustive
validity check can also be made to every single part produced.
Continuous advances have been made in sensing technologies and,
particularly, in the vision sensors that have been specially
enhanced in capabilities with lower cost. The advances made in
the image processing technology also provide more reliable
solutions than before. In all, computer vision is a very useful
non-invasive technique for the industrial environment. The use of
these systems in other monitoring operations in machining
processes has proved [5,18] an important reduction in the cycle
time and the resources. In this field, two guidelines should
be remarked: the study in spatial domain and in frequency
domain
[56,133]. Indirect
tool
condition
monitoring
using
image
Table 1 (Continued )
Researcher Illumination sys-
tem
Image processing Type of tool wear
measurement
Machining Remarks
Duan et al. [30] Front lighting with
LED
Histogram generation, level set
based contour segmentation,
histogram based contour
segmentation, fusion of both
segmentation, wear
measurement
Flank wear
detection for
progressive
machining
Micro drill-bit
(for PCB drilling)
Capable to remove the noise
due to smearing; More
computation time
Xiong et al. [135] Fluorescent high
frequency linearlight
Variational level set based
segmentation, no need for re-initialization of zero level set
Tool wear area Milling inserts No measurement of flank
wear width
Otieno et al. [96] Dome light with
low intensity back
lighting
Histogram equalization,
Gaussian filtering, XOR
operation for edge detection
Micro-Milling tool No measurement of wear
Yasui et al. [138] Microscope Thresholding, edge detection to
segment the wear flats from its
background
Grinding wheel
wear
Grinding Accuracy is low, possibility
for detection of false wear
flats
Lachance et al. [75] Fibre optic guided
light with beam
splitter
Thresholding, region growing Progressive wear of
grinding wheel
Grinding Morphological operations
will be lead to more accurate
segmentation
Prasad and
Ramamoorthy [100]
White light Histogram, GLCM and fractal-
based texture analysis
Progressive wear of
grinding wheel
Grinding Simple, faster but less
accurate
Karthik et al. [61] Automatic focusing
at various height
(interpolation
and
search technique
for
improvingaccuracy)
Image consolidation, median
filtering, thresholding, laplacian
contour
detection,
edge
linking,
dilation, chain coding, MLPNN
for
classification
Flank and crater
wear (depth)
measurement
and
classification
Turning inserts Leads to 3D measurement;
flank wear, crater wear,
chipping
and
breakage
were
classified; 3D map for crater
wear
has
not
been
evaluated;difficult
for
grooved
inserts
Prasad and
Ramamoorthy [100]
Stereo vision
technique using
law of triangulation
Stereo image processing for
getting the 3D map of crater,
MLPNN
Flank wear and
crater wear
prediction and
progressive wear
measurement
Turning inserts Less accurate technique for
crater depth less than
125mm; no technique to
reduce the noises from dirt,
dust, oil etc. difficult for
grooved inserts
Devillez et al. [24] White light
interferometer
White light interferometry by
automatic and varying focusing
Crater depth
measurement
Inserts Difficult to measure grooved
inserts
Dawson and
Kurfess
[22]
White light
interferometer
Volume reduction measurement
of
tool
from
fusion
of
CAD
model
and surface profile
Crater depth
measurement
Inserts Difficult to measure grooved
inserts
Wang et al. [130] LCD projector for
fringe creation on
rake surface
3D reconstruction using phase
shifting method from 4 fringe
patterns with 4 phase shifting
angle
4 parameters of
crater wear
measurement
Inserts Difficult to measure grooved
inserts
S. Dutta et al./ CIRP Journal of Manufacturing Science and Technology 6 (2013) 212232 221
8/10/2019 Tool Monitoring Using Image Processing
11/21
8/10/2019 Tool Monitoring Using Image Processing
12/21
tried tomonitor the condition of a sharp, a semi-dull and adull tool
by this technique. However, they did not analyze the error of
prediction. Kassim et al. [64] introduced a procedure to define
edges of surface texture obtained from turning, end milling and
face milling operation by connectivity oriented fast Hough
transform parameters like spread of orientation, average line
length, main texture orientation and total fitting error. This
connectivity oriented fast Hough transform process was faster and
less computationally complex than standard Hough transform
technique which was used to analyze the uniformity of surface
textures obtained from sharp and dull tools. Then the tool wear
was then predicted by using a MLPNN where inputs were taken
from the parameters of processed images. However, they did not
get any correlation for image number 35. Kassim et al. [63] also
showed that run length statistics technique for the detection of
surface textures machined by sharp tool and dull tool was faster
and better than column projection technique and connectivity
oriented fast Hough transform technique. Column projection
analysis technique was working well for highly regular surfaces
whereas Hough transform technique was extracting line segments
for variety of length. With the features extracted from run length
matrix, they classified the sharp tool and dull tool by applying
Mahalanobis distance classifier. Also they compensated inhomo-
geneous illumination of the texture images through an excellentway. However, they did not get any systematic trend of variation
between image textureparameters andmachining time. The image
descriptors were not normalized and no correlation study of image
texture descriptors with progressive tool wear or surface
roughness has been indicated in their work.
In a very recent study, Datta et al. [21] captured the turned
surface images for progressive wear of a uncoated carbide tool and
analyzed those images using a grey level co-occurrence matrix
(GLCM) technique based texture analysis. They also find a linear
correlation between the extracted features, namely, contrast and
homogeneity with the tool wear in terms of slope of the linear fit
and a fitting parameter, coefficient of determination. It has also
been observed from their study that the selection of GLCM
parameters viz. pixel pair spacing and direction is very muchimportant to get the accurate results as the distribution of feed
marks are varyingwith the variation of machining conditions (feed
rate and depth of cut). However, they did not mention about any
method to find the optimum pixel pair distance. As an improve-
ment of theprevious technique, Dutta et al. [31]has beenproposed
a novel technique to find the optimum pixel pair spacing
parameter to get an accurate resultby textureanalysis ofmachined
surfaces with the progressive tool wear. They got a periodic
relation of extracted texture descriptors viz. contrast and
homogeneity with the different pixel pair spacing. Utilizing this
periodic property, they found out the periodicity using Fourier
power spectral density technique and later on they found the
optimum pixel pair spacing parameter of GLCM. However,
the
optimum
pixel
pair
spacing
is
also
varying
dependent
onthe change of feed rate. They got a very good correlation of
extracted descriptors with tool wear and surface roughness.
However, they did not do any experiment to detect the progressive
tool wear of coated carbide tools.
Fractal analysis of surface texture for tool wear monitoring was
proposed by Kassim et al. [66] to deal with high directionality and
self-affinity of end-milled surfaces and a hidden Markov model
(HMM) was used to differentiate the states of tool wear.
Anisotropic nature of end-milled and turned surface textures
was analyzed by fractal analysis along different directions to the
entire image by Kassim et al. [65]. They used a 13-element feature
vector to train the HMMmodel for classifying fourdistinct states of
tool condition. However, no estimation of classification error has
been
encountered
in
their
study.
Kang
et
al.
[60]
used
a
fractal
analysis technique to study the variation of fractal dimension with
measured surface roughness, wear values with machining time for
different feed combination for high-speed end milling of high-
hardened material by a coated carbide tool. However, no
quantitative analysis of correlation of fractal dimension with
flank wear or surface roughness was done.
Persson [98] established a non-contact method to measure the
surface roughness by incorporating angular speckle correlation
technique. A speckle pattern created on the machined surface with
the help of a coherent HeNe laser and captured at different angle
of illumination. Then a correlationbetween those captured speckle
pattern at different angle of illumination has been calculated. The
lower correlation value has been observed for rougher surfaces.
Though this technique canbeused for the in-process measurement
of surface roughness but the accuracy of this method is limited by
the proper angular positioning of the set-up. However, this
limitation can be overcome by using a laser interferometric
technique for tilt measurement of the set up.
With a different approach, Li et al. [79] has been introduced an
waveletpacket analysis of machined surface images obtained from
turning operation. They got a good correlation between the
extracted feature, namely,high frequency energy distribution ratio
with progressive cutting tool wear. However, a systematic
quantitative correlation analyses was missing in their study.
5.2.
Offline
techniques
Luk and Huynh [85] analyzed the grey level histogram of the
machined surface image to characterize surface roughness. They
found the ratio of the spread and the mean value of thedistribution
to be a nonlinear, increasing function of Ra. Since their method was
based solely on the grey level histogram, it was sensitive to the
uniformity and degree of illumination present. In addition, no
information regarding the spatial distribution of periodic features
could be obtained from the grey level histogram. Hoy and Yu [45]
adopted the algorithm of Luk and Huynh [85] to characterize the
surface quality of turned and milled specimens. They found one
exception where the ratio of the spread and the mean of the grey-level distribution was not a strictly increasing function of surface
roughness and, therefore, the value of the ratio might lead to
incorrect measurement. They also addressed the possibility of
using the Fourier transform (FT) to characterize surface roughness
in the frequency domain. However, only simple visualjudgement
of surface images in the frequency plane was discussed. No
quantitative description of FT features for the measurement of
surface roughness was proposed. Al-kindi et al. [7] examined the
use of a digital image system in the assessment of surface quality.
The measure of surface roughness was based on spacing between
grey level peaks and the number of grey level peaks per unit length
of a scanned line in the grey level image. This 1D based technique
did not fully utilize the 2D information of the surface image, and is
sensitive
to
choice
of
lay
direction,
lighting
and
noise.
Cuthbert
andHuynh [20] increased the sophistication of the analysisby applying
a statistical texture analysis on the optical Fourier transform
pattern createdon the ground surface images.Then they calculated
the mean, standard deviation, skewness, kurtosis, and root mean
square height of the grey level histogram of the image. There were
two limitations of this technique. Only surfaces upto an average
surface roughness of 0.4 mm could be inspected, as rougher
surfaces tend to createadiffusedpattern in the camera. Precise and
complex alignment of the imaging optics was required, thereby
making it difficult to the use in online inspection.Jetley and Selven
[54] used the projection of a reflection pattern of a beam of low
power (1 mW) HeNe laser light from ground surface. Then the
pattern was analyzed and characterized using blob area, thresh-
olding
and
hence
correlated
to
the
surface
roughness.
But
the
S. Dutta et al./ CIRP Journal of Manufacturing Science and Technology 6 (2013) 212232 223
8/10/2019 Tool Monitoring Using Image Processing
13/21
8/10/2019 Tool Monitoring Using Image Processing
14/21
surface images using the GLCM texture descriptors. However, they
have not optimized any of the GLCM parameters. Also they have
only tested this method for milled surface images only. Gadel-
mawla [39] predicted average surface roughness (Ra) values from
the texture descriptors extracted from the GLCM of turned surface
images with only a single combination of GLCM parameters for
different machining conditions. The error between the measured
Ra value by stylus method and the predicted Ra value is 7%.
However, the distance parameter of GLCM could be optimized for
getting
more
accurate
and
precise
result.
Myshkin et al. [89] introduced a special type of co-occurrence
technique with the concept of multi-level roughness analysis
to determine the surface roughness for nanometer scale
deviations obtained from the atomic force microscope (AFM)
images. However, no quantitative analysis has been done in their
study.
Tsai et al. [123] investigated Fourier power spectrum of shaped
and milled surface images with various maximum surface
roughness. The maximum surface roughness values were mea-
sured using a stylus-based surface profiler. They found image of
the surface patterns of the shaped specimens were more regular
and present less noise than those of the milled specimens. They
further found a monotonically decreasing trends for feature major
peak frequency, principal component magnitude squared, centralpower spectrum percentage and monotonically increasing trends
for average power spectrum with increasing values of measured
surface roughness for both the shaped and milled parts. Further-
more they used two artificial neural network (ANN) techniques for
classification of roughness features in fixed and arbitrary orienta-
tions of surfaces. Then they selected major peak frequency as the
best feature for both shaped and milled specimen in fixed
orientation, because, it was the distance between the major peak
and the origin, so it was a robust measure to overcome the effect of
lighting of the environment. However, they only did the surface
finish measurement for flat surfaces not for curved parts. Tsai and
Wu [124] used a Gabor filter-based technique for an automated
classification of defective and non-defective surfaces from the
surface images. They convolved the image with a 2D Gaborfunction, which is an oriented complex sinusoidal grating
modulated by a 2D Gaussian function. Then they have selected
the best parameter of the Gabor function, such that the energy of
the convolved image was zero, using exhaustive search method.
Then a threshold value has been chosen using statistical control
method for distinguishing the homogeneous and non-homoge-
neous surface texture. However, a very accurate controlled set-up
for capturing the surface images are required for practical
accomplishment of their method. Dhanasekar et al. [25] captured
speckle patterns of machined surfaces (ground and milled) using a
collimated laser beam (HeNe laser, 10 mW, l = 532 nm) and a
CCD camera. Then, pre-processing of speckle images was carried
out to remove unwanted intensity variations due to ambient
lighting
change,
etc.
The
speckle
images
were
filtered
by
Butter-worth filter and then the centralized fast Fourier transform (FFT)
was determined. After that average and integrated peak spectral
intensity coefficient and autocorrelation coefficient in X, Y and
diagonal directionswere determined. The width of autocorrelation
functions for smooth and rough images were varied. The spectral
speckle correlation (auto-correlation) technique for surface
roughness assessment had been used before and after pre-
processing of speckle images. They were then compared to stylus
values (Ra). It was found that autocorrelation parameters after pre-
processing had a better correlation (i.e. higher correlation
coefficient) with the average surface roughness (Ra) measured
for themilledand ground components. To getmore accurate result,
image model for compensating inhomogeneous illumination [14]
could
be
used
in
their
work.
Josso et al. [57] analyzed and classified eight surface images
obtained from eight types of engineering processes viz. casting,
grinding, gritblasting, hand filing, horizontal milling, linishing,
shotblasting, vertical milling. They have developed a space-
frequency representation of surface texture using frequency
normalized wavelet transform (FNWT) and extracted some surface
finishdescriptors. Then they classified those eight types of surfaces
using discriminant and cluster analysis approach. However, there
is a high chance of misclassification between similar types of
texture viz. milling and grinding. So, they compared continuous
wavelet transform (CWT), standard and scaled discrete wavelet
transform (DWT) methods and concluded that the standard
discrete wavelet transform associated with cluster analysis was
the best method for classification purpose. In their another work
[55], they tried to measure the form, waviness and roughness of
machined surfaces images by using FNWT. Niola et al. [94] tried to
reduce the problem of brightness variation on surface images at
different lighting condition by enhancing images of machined,
ground and polished surfaces using Haar wavelet transform.
However, no surface finish descriptors were extracted from the
surface images, in their study.
Ramana and Ramamoorthy [104] classified ground, milled and
shaped images based on GLCM, amplitude varying rate approach
and run length statistical technique. However, they did not decideabout the best feature for vision-based surface roughness
measurement. Also they did not do any quantitative correlation
study between vision based and stylus based surface roughness.
Bradley and Wong [16] presented the performance of three image-
processing algorithms, namely, analysis of the intensityhistogram,
image frequency domain analysis and spatial domain surface
texture analysis for evaluating the tool condition from face milled
surface images. Though, the histogram based technique revealed a
proper trend for the progressivewear of face milling tool but itwas
very much influenced by the lighting condition. Frequency domain
technique was much less sensitive to inhomogeneous illumination
than the histogram based approach. The major advantage of a
texture-based method was the dependence on localized similari-
ties in the image structure. The absolute value of illuminationintensity was not critical; the illumination must be sufficient to
highlight image features. Similarly, the method was not sensitive
to the angle of illumination, except for extreme cases where the
axis of illumination approached 908. They showed a systematic
variation of texture parameters with machining time. However, no
quantitative correlation has been reported by them. Zhang et al.
[142] developed an accurate defect detection and classification
system by extracting the best features from discrete cosine
transform (DCT), Laws filter bank, Gabor filter bank, GLCM. They
used support vector machine (SVM) and RBFNN for classification
purpose. They have got a 82% success using the combination of
Gabor filter, GLCM and SVM. Singh and Mishra [115] classified
different types of spangles obtained due to the galvanization of
steel
sheets
using
GLCM
and
Laws
texture
descriptors
with
RBFNN.They achieved 80% accuracy of classification. Their approach can
also be used for progressive wear monitoring. Alegre et al. [4] used
first order statistical texture analysis, GLCM method and Laws
method to evaluate turned surface images and classified two
roughness classes using k-NN technique. Best result was obtained
by using Laws method, in their study. In a different approach,
Bamberger et al. [12] compared three methods for examining the
chatter marks produced at the time of machining in valve seat of
automotive parts from the images of the valve seats. They
compared three image processing based techniques, namely, circle
fitting, circularity and GLCM method to classify accepted and
rejected parts. Though, they selected the appropriate distance
parameter of GLCM, manually, but it is needed to develop an
automatic
method
for
detection
of
optimized
distance
parameter.
S. Dutta et al./ CIRP Journal of Manufacturing Science and Technology 6 (2013) 212232 225
8/10/2019 Tool Monitoring Using Image Processing
15/21
Table 2
Indirect TCM techniques based on image processing.
Researcher Illumination system Image processing algorithm Applied in Remarks
Wong et al. [134] HeNe laser Mean and standard deviation of
laser pattern created on machined
surface
Turning Offline; no study on correlation
and progressive wear
Gupta
and
Raman
[42]
HeNe
laser,
circular
variable attenuator
Histogram
based
1st
order
statistical texture analysis
Turning
(moving
and
static condition); surface
roughness measurement
Online;
no
correlation
study
between vision-based surface
roughness and stylus-based
surface
roughness
andprogressive tool wear; no
discussion about blurring due to
movement
Tarng and Lee [119] 2 Light sources
situated at an acute
angle with the axis of
workpiece
Determination of Ga, polynomial
network with self organized
adaptive learning (feed, speed,
depth
of
cut
and Gaas input, Raas
output)
Turning; Raprediction Online; prediction error
(max)=14%; extraction of 1
descriptor only; no prediction of
tool
wear
Ho et al. [44] 2 Light sources
situated
at
an
acute
angle with the axis of
workpiece
Determination of Ga, ANFIS (feed,
speed,
depth
of
cut
and Gaas input,
Raas output)
Turning, Raprediction Online; prediction of Rausing
ANFIS
prediction
error
(max)=4.55%; extraction of 1
descriptor only; no prediction of
tool wear
Lee et al. [78] A diffused blue light in
458 inclination
Standarddeviation of grey level, two
frequency domain parameters and
abductive network (input as 3
texture descriptors, output as Ra)
Turning, Raprediction Online; max prediction
error=14.96%; no prediction of
tool wear
Lee et al. [79] A diffused blue light in
458 inclination
Standarddeviation of grey level, two
frequency domain parameters and
ANFIS
(input
as
3
texture
descriptors, output as Ra)
Turning, Raprediction Online; max prediction
error=8%; no prediction of tool
wear
Akbari et al. [3] Scattered pattern of
light
Histogram based 1st order
statistical texture analysis (four
descriptors) & MLPNN
Milling, Raprediction Online; No quantification of
prediction error; No prediction
of tool wear
Narayanan et al. [91] An evolvable hardware Image enhancement, determination
of Ga, genetic algorithm
Milling; Surface
roughness measure
Online; no quantification of
prediction error; no prediction of
tool wear
Sarma et al. [110] Determination of Ga, frequency
domain analysis
Turning GFRP composite
with PCD tool
No study for progressive wear
monitoring
Palani
and
Natarajan
[97]
Frequency
and
spatial
domain
based
texture analysis, BPNN
End
milling, Raprediction No study for progressive wear
monitoring
Kassim et al. [67] Sobel operation, thresholding,
column
projection
(CP)
(applied
on
thresholded images), run-length
statistics (RLS) (applied on greylevel images)
Turning; progressive wear
monitoring
Online; Progressive wear
monitoring;
classification
between sharp tool and dull tool
in various machining; nocorrelation study with Ra
Mannan et al. [87] Sobel operation, thresholding, CP,
RLS, extraction of AE parameters
using wavelet analysis, RBFNN for
flank
wear
prediction
Turning; progressive wear
monitoring
Online; monitor sharp, semi-dull
and dull tool; no quantification
of prediction error
Kassim et al. [64] Canny edge detection, connectivity
oriented fast Hough transform,
MLPNN
for
FW
prediction
Turning, end milling, face
milling; progressive wear
monitoring
Online; no quantification and
comparison of prediction error
Kassim et al. [63] Compensating inhomogeneous
illumination compensation,
comparison of CP, connectivity
oriented fast Hough transform and
RLS, Mahalanobis distance classifier
for classification of sharp and dull
tool
Turning; progressive wear
monitoring and
classification
Online; RLS was selected as the
best technique depending only
on a single cutting condition;
classification between two wear
state only; more
experimentation needed
Datta et al. [21] Diffused light GLCM technique Turning; progressive wear
monitoring
Online; extraction of best feature
depending
only
on
a
singlecutting
condition;
No
optimization of GLCM
parameters
Datta
et
al.
[31]
Diffused
light
GLCM
technique
with
optimized
pixel pair spacing (pps) parameter
Turning;
Progressive
wear
monitoring
Online;
Optimization
of
pps
developed; applicable for any
periodic textures; no study to
monitor coated carbide tool
Kassim et al. [66] Fractal with HMM End milling; Classification Online; No estimation of
classification error
Kassim et al. [65] 3D fractal with HMM End milling; classification
of 4
states
of
wear
Online; no estimation of
classification
error
Kang et al. [60] Fractal; progressive variation study
with surface roughness and tool
wear
High speed end milling
(with coated carbide)
Online; no estimation of
correlation parameter
Li et al. [81] Diffused light Wavelet packet decomposition Turning; progressive wear
monitoring
Online; no correlation analysis
with tool wear
S. Dutta et al./CIRP Journal of Manufacturing Science and Technology 6 (2013) 212232226
8/10/2019 Tool Monitoring Using Image Processing
16/21
Table 2 (Continued )
Researcher Illumination system Image processing algorithm Applied in Remarks
Hoy and Yu [45] Diffused white light Histogram analysis, 2D FFT analysis Turning, milling Offline; no progressive wear
monitoring
Cuthbert and Hynh [20] HeNe laser, spatial
filter, beam splitter
and mirror
Histogram based 1st order
statistical texture analysis
Grinding Offline; complex attenuator;
difficult to implement for high
roughness values; no
progressive wear monitoring
Jetley and Selven [54] HeNe laser Blob analysis, thresholding Grinding Offline; no progressive wear
monitoring
Ramamoorthy andRadhakrisnan [103]
GLCM analysis Grinding, shaping, milling Offline;no correlationparameterstudy
Kiran et al. [71] Diffused light; light
sectioning; phase
shifting with grating
projection
Frame averaging; low pass filtering;
2nd order co-occurrence statistics;
three lighting methods were
compared for rough, medium rough
and smooth images
Grinding, milling, shaping Offline; mainly the comparison
of three types of lighting; no
roughness evaluation
Younis [140] White light Neighbourhood processing Grinding (different
material)
Offline; coefficient of variation
8.6%
Coefficient of determination (R2)
0.790.92; no progressive wear
study
Kumar et al. [72] Cubic convolution interpolation,
linear edge crispening,
Determination of Ga
Shaping, milling, grinding Offline; no progressive wear
monitoring
Khalifa
et
al.
[69]
Edge
enhancement,
magnification,
statistical texture analysis (1st and
2nd
order),
calculation
of
Gavalue
Chatter
detection
in
turning
Discrimination
between
chatter-
rich and chatter-free process
from
surface
imagesAl-kindi
and
Shirinzadeh [8]
Ambient
light
Comparison
between
two
lighting
models viz. intensity topography
compatibility and light diffused
model, extraction of optical surface
roughness parameters from 1st
order statistics
Face
milling
No
correlation
study
with
progressive wear
Elango and
Karunamoorthy [32]
Diffused light at
different grazing angle
Determination of Ga, Taguchis
orthogonal array and ANOVA
Face turning No correlation study with
progressive wear
Dhanasekar and
Ramamoorthy [28]
White light POCS for reconstruction of high
resolution image, frequency domain
and
histogram
based
texture
analysis, GMDH
Milling, grinding (Raprediction)
No prediction error analysis, no
correlation study with
progressive
wear
Zhongxiang et al. [143] Stereo zoom
microscope, halogen
lamp
Median filtering, histogram
conversion, histogram
homogenization, calculation of 3D
roughness
Paning, plain milling, end
milling, grinding
No correlation study with
progressive wear
Dhanasekar and
Ramamoorthy [26]
RichardsonLucy algorithm for
deblurring, frequency and spatial
domain based texture analysis, ANN
Milling, grinding, Raprediction
Correlation coefficient 0.923 and
0.841 for milling and grinding,
No correlation study with
progressive wear
Gadelmawla[36]
Microscope
GLCM,
study
the
effect
of
pps
Face
turning
No
optimization
ofpps value, No
correlation study with
progressive wear
Gadelmawla
et al. [37,38]
Microscope
GLCM
Milling,
Reverse
engineering for cutting
conditions
No
optimization
ofpps value, No
correlation study with
progressive wear
Gadelmawla [39] Microscope GLCM Face turning, Correlation
with Ra
No optimization ofpps value, No
correlation study with
progressive wear
Tsai et al. [123] Fluorescent light
source
Fourier analysis, ANN Shaping, Milling No correlation study with
progressive wear
Tsai and Wu [124] Gabor filtering, classification of
defective and non-defective parts,
Milling No mention of success rate; no
progressive wear or surface
roughness
study
Dhanasekar et al. [25] HeNe laser Speckle pattern, Butterworthfiltering, Fourier analysis,
Autocorrelation
Grinding, milling No correlation study withprogressive wear
Josso et al. [57] Frequency normalized wavelet
transform, discriminant and cluster
analysis
Classification of ground,
milled, cast surfaces etc.
No correlation study with
progressive wear
Josso et al. [56] Frequency normalized wavelet
transform,
Form, waviness,
roughness measurement
No correlation study with
progressive wear
Niola et al. [94] Haar wavelet for reduction of
inhomogeneous
illumination
Milling, grinding,
polishing
No extraction of surface finish
parameters
Raman
andRamamoorthy [104]
GLCM, amplitude varying rate
method, RLS
Classification of ground,
milled, shaped surfaces
No correlation study with stylus
based surface roughness; no
progressive
wear
study
S. Dutta et al./ CIRP Journal of Manufacturing Science and Technology 6 (2013) 212232 227
8/10/2019 Tool Monitoring Using Image Processing
17/21
Table 2 (Continued )
Researcher Illumination system Image processing algorithm Applied in Remarks
Bradley and
Wong [16]
Fibre optic guided light
(regulated)
Frame averaging, Gaussian filtering,
median filtering, after filtering:
image histogram analysis,
frequency domain analysis, texture
segmentation
Face milling, progressive
wear study
Comparison between histogram
analysis, frequency domain
analysis and texture
segmentation; no correlation
analysis of vision-based surface
finish with tool wear
Zhang et al. [142] DCT, Laws filter, Gabor filter, GLCM,
Shape features, SVM with RBFNN
kernel
Defect detection and
classification in ground
and polished surfaces
82% success rate using the
combination of Gabor filter and
GLCM with SVMAlegre et al. [4] DC regulated lightwith
SCDI
First order statistical texture
analysis, GLCM, Laws method, k-NN
classification
Turning No progressive wear study
Nakao [90] Fibre optic light Thesholding, component labelling Drilling burr
measurement
3% and 2% error in measuring
burr thickness and height
Yoon and
Chung [139]
Halogen (front light)
LED (back light)
Edge detection (burr width
measurement), Shape from focus
(burr height measurement)
Micro-drilling 0.1mm resolution; less than
0.5mm accuracy
Sharan and
Onwubolu [114]
High intensity spot
lighting
Burr profile measurement Milling 2.2mm resolution
Fig. 3. Flow diagram of proposed tool condition monitoring technique using digital image processing.
S. Dutta et al./CIRP Journal of Manufacturing Science and Technology 6 (2013) 212232228
8/10/2019 Tool Monitoring Using Image Processing
18/21
Ikonen and Toivanen [46] proposed an algorithm that gave
priority to a pixel in the tail so as to calculate the minimum
distance in a curved space so that it helped in calculating the
roughness in a faster and more efficient manner.
Vesselenyi et al. [126] utilized 2D box counting method and
found nine parameters as roughness descriptor by linear, second
order and third order polynomial fitting on shaped, ground and
polished surface images of different surface roughness. Then
they classified them using C-means clustering. However, more
number of sampleswere required to proof the suitability of their
method.
Quality of honed surfaces was also determined by Leon et al.
[80] using image processing technique. He quantified the groove
textures anddefects of honed cylinder bore in frequency domain.
In frequency domain, the groove texture of interest was
separated from the other defects such as groove interrupts,
holes, cracks, flakes, material defects, graphite lamellae, material
smearings, smudgy groove edges and foreign bodies. The images
were taken from fax film replicas of honed surfaces. The images
were enhanced by contrast stretching. Digital image processing
was also used in chatter identification and burr detection in
machining.
Nakao [90] captured images of drilling burrs and then
processed to monitor drilling process. Here the conventionalimage processing techniques such as the binary image proces-
sing, the noise reduction and the labelling were applied to
measure image data. Here burr height and thickness were
measured fromthe processed image usingco-ordinate data. Yoon
et al. [139]usededge detection algorithm tomeasurehole quality
and burr width in micro-drilled holes. They also measured burr
height with Shape From Focus (SFF) method. Here a halogen
light was used as a front light and LED was used as a backlight for
getting uniform illumination. Sharan and Onwubolu [114]
measured the burr profile of milled parts with 2.2 mm system
resolution.
In most of the research, the variation of vision based surface
texture descriptors with machining time were not studied for
progressive wear monitoring. Also there is a requirement tonormalize the texture or wear descriptors for reducing the effects
of lighting variations. Research in this area is requiring a detailed
study with various work tool material combination with various
cutting parameters for differentmachining application to establish
a robust monitoring system.
The indirect tool condition monitoring techniques, using image
processing are summarized in Table 2.
6. Conclusions
In this paper, the application of image processing technology
applied for tool condition monitoring is discussed. For real time
tool condition monitoring with noncontact techniques, the image
processing
algorithms
can
be
used
for
enhancing
the
automationcapability in unmanned machining centres.
The digital image processing techniques are very useful for fast
and easier automatic detection of various types of tool wear (such
as crater wear, tool chipping and tool fracture) which are very
difficult to recognize by other modes. Textural analysis techniques
are playing a predominant role for tool condition monitoring via
assessment of ma