Knowledge-based line-correction rules in a machine-vision system for the leather industry

  • Published on

  • View

  • Download

Embed Size (px)


<ul><li><p>Engng Applic. Artif. Intell. Vol. 4, No. 6, pp. 433-438, 1991 Printed in Great Britain. All rights reserved </p><p>Contributed Paper </p><p>0952-1976/91 $3.00 + 0.00 Copyright 0 1991 Pergamon Press plc </p><p>Knowledge-based Line-correction Rules in a Machine-vision System for the Leather Industry </p><p>ATTILA LERCH Hungarian Academy of Sciences, Budapest </p><p>DMITRY CHETVERIKOV Hungarian Academy of Sciences, Budapest </p><p>LeaVis (for Leather Vision) is a machine-vision system that provides visual input for a CAD system used in the leather industry. It has recently been developed in the Computer Vision and Robotics Laboratory of the Computer and Automation Institute. Its task is to process and segment images of hides that are manually marked by lines and other symbols. A key feature of the vision system is its ability to correct line drawings by connecting broken lines and bridging gaps in line junctions. The correction is guided by a set of knowledge-based rules that allow the program to handle the contours in a consistent way. The paper gives an overview of LeaVis and discusses the mechanism of line correction in more detail. </p><p>Keywords: Image analysis, industrial application, knowledge-based rules, line correction. </p><p>THE TASK OF THE VISION SYSTEM </p><p>In the shoe and upholstery industry, large hides are cut into pieces of desired shapes, e.g. parts of shoes. Normally, a hide has defects and areas of different quality suitable for different pieces. A limited number of expensive computer systems exist that help to auto- mate the visual input of a hide image in order to optimize the cutting process by using a computer-aided layout design system and a programmable knife. </p><p>LeaVis (for Leather Vision) is a machine-vision system for the analysis of hide images which has recently been developed at the Computer and Automation Institute in cooperation with VICAM, a leading Austrian company producing CAD software for the shoe and upholstery industry. In the current setup, a human operator manually draws on the hides the borders of the quality areas (Q-areas), marks the areas by specially designed stamps (called quality marks, or Q-marks) and indicates the defects. The vision system acquires an image of the marked hides, processes and corrects the line drawing, recognizes the stamps and segments the image. A polygonal represen- tation of the quality areas is passed to a CAD system. </p><p>Full automation of the process is a challenging but </p><p>Correspondence should be sent to: A. Lerch, Computer and Automation Institute, Hungarian Academy of Sciences Budapest, P.O. Box 63, H-1518 Hungary. </p><p>extremely difficult problem. Manual marking is needed because the image of a hide surface does not contain all the information that is necessary to determine the quality areas. Some other factors such as the local thickness of the hide must also be taken into account. </p><p>The manual indication of the quality areas simplifies the task of the vision system, but a few serious prob- lems still remain. Hide images are quite noisy. A key problem is the quality of the manual drawing which is far from being perfect. There are gaps in lines and line junctions that reach 20-30 pixels. Because of the limited capacities of the hardware used and the very high resolution required, line correction must be achieved in software. Hide images whose size is approx 7000 x7000 pixels must be coded and compressed at the very beginning of the processing. On the other hand, the execution time of the program is limited by the manufacturing process. </p><p>An image may contain several hides of similar col- our. Touching hides are treated as one hide. The outer contours of hides, including possible holes, have also to be found. Figure 1 shows a binarized test image of a hide. The manually drawn lines define regions of uniform quality and defect areas. The lines have gaps as illustrated by Fig. 2. A quality area border may partly be formed by the outer contour of a hide. Q-areas may be nested in arbitrary depth. Each Q-area except the defect (zero quality) areas is marked by a quality mark. </p><p>433 </p></li><li><p>434 ATTILA LERCHandDMITRYCHETVERIKOV:MACHINE-VISIONSYSTEM </p><p>Fig. 1. A binarized test hide image with quality marks </p><p>Fig. 2. An example of broken lines and gaps in line junctions. </p><p>A Q-mark is placed inside the appropriate Q-area. Areas without Q-marks are zero quality, i.e. unusable areas. For proper assignment of quality marks to qua- lity areas, the contours of these areas should be closed in spite of gaps in lines and line junctions. </p><p>OUTER CONTOUR EXTRACTION </p><p>PRINCIPLES OF OPERATION OF LeaVis </p><p>The main hardware components of the LeaVis system are the mechanical scanning table carrying a 3 x 4 m size palette, the frame with a fixture for the cameras, two top-view high-resolution line-scan CCD cameras and a PC AT (386) equipped with the image digitizer and processor card. Two slightly overlapping line-scan cameras with a total resolution of approx 7000 pixels per line are used to obtain the required precision of about 1 mm. </p><p>Figure 3 illustrates the operation of LeaVis. The system scans a hide, processes and encodes its digital image and approximates the contours. The main branch of the image analysis program recognizes the quality marks, corrects the line contours, assigns Q-marks to the proper Q-areas and compiles the description of the hide required by the CAD system. A separate branch of the program extracts the outer contours of the hide. </p><p>The Image Digitizer and Processor yield an image with eliminated (shrunk) lines and an image with enhanced lines. Both images are binary. The former is used for the outer contour extraction, while the latter serves for the segmentation purposes. The Run-Length Coder computes the run-length code of the image during the scanning. The Chain Coder transforms the run-length code into a sequence of chain codes describ- ing the contours of connected components of lines. After the chain coding, all objects in the image, e.g. pieces of lines, are represented by closed contours. The Polygonal Approximator yields a polygonal approxima- tion of the chain-coded contours. </p><p>In the main branch of the program, the Turnpoint Finder detects the ends of lines for subsequent line correction. The Quality Mark Recognizer recognizes quality marks of specific shapes. The Line Corrector tries to fix the line defects resulting from poor drawing, image noise, or illumination effects. The Quality Assignment Routine assigns quality marks to the proper quality areas that may be nested. Figure 4 demonstrates a segmentation of the hide image shown in Fig. 1. </p><p>In the other branch of the program, the Outer </p><p>EIlh. lines Poly. Turn. Q-mark Line Q-mark Recognized </p><p>Awrox. Point Recoqn. Corr. Assicrn. Q-areas I </p><p>SEGMENTATION I </p><p>Recognized Q-mark contours </p><p>Fig. 3. LeaVis: principles of operation. </p></li><li><p>ATTILA LERCH and DMITRY CHETVERIKOV: MACHINE-VISION SYSTEM </p><p>Contour Extractor uses a shrunk-line image to extract the outer contours of the hides. A more-detailed description of the operation of LeaVis can be found in Ref. 1. In the rest of this paper, a line correction algorithm is presented that significantly improves the reliability of the vision system and makes its practical application possible. </p><p>LINE CORRECTION </p><p>The basic idea behind all types of error correction is making use of some sort of a priori knowledge about the phenomenon under consideration, in order to eli- minate distortion and noise in the data. This applies to line correction as well. Line drawings are composed of objects whose elements are continuous line segments of curves. The objects themselves vary from task to task. For example, in the case of character recognition they are characters, in this case quality area borders. In general, correction is aimed at restoring the continuity of an object which is broken into several segments, or separating fused objects. For this purpose, one has to possess information about the characteristics of the objects. </p><p>The correction itself is generally made in two steps: first, locating characteristic points of objects or their elements; second, applying rules based on a priori knowledge to guide the appropriate connection and separation operations. The characteristic points can be, for instance, those points where the curvature of the contours is large, or changes its sign (i.e. corner or inflexion points). As a rule, correction operations are based on these interest points. </p><p>Let us now discuss the characteristic points and knowledge-based rules used for line correction in the LeaVis system. A hide image is partitioned by manually drawn lines into regions (quality areas) that may be nested. These physical lines are the borders of the regions. After the polygonal approximation, each open line segment is represented by a polygon sur- </p><p>Fig. 4. The recognized quality areas of the hide shown in Fig. 1. </p><p>EM1 4:6-6 </p><p>Fig. 5. Hypothetical hide contours approximated by polygons and their tracking directions </p><p>rounding the physical line area (see Fig. 5). A closed curve is represented by two polygons, one for the inner, the other for the outer contour of the curve. To dis- tinguish between the two types of polygons, a tracking direction is assigned to each polygon. The inner con- tours are tracked clockwise, and the outer ones counter-clockwise. An open line segment is repre- sented by a counter-clockwise polygon. Analogously, consistent rules are applied to the background (palette), which is binarized to the same value as the lines. Note that after the binarization there is no direct way to distinguish between the lines and the back- ground. </p><p>In the case of marked hides, there is practically no need for the separation of fused objects since touching lines or other symbols occur very rarely. Such errors cannot be effectively treated in the framework pre- sented in this paper, and they will not be considered. Also, it should be mentioned that it is impossible to guarantee automatic correction of all errors that can appear on hides. Therefore, a possibility of contour checking and interactive post-correction is provided in the CAD system as an option for exceptional cases, such as very large gaps in poorly drawn lines. </p><p>Line end detection </p><p>The line correction procedure relies on the proper definition and location of characteristic points on the hide contours represented by polygons. Usually, inflex- ion points and points of maximum curvature are con- sidered that provide significant information about curves. A number of algorithms have been developed to find the so-called corner points on a contour. In our case the notion of significant points can be simpli- fied: it is sufficient to consider only those points where line segments terminate. The reason why no other characteristic points are needed is that the correction algorithm is not concerned with the separation of objects. In terms of contour polygons, line termination means a drastic change of polygon edge orientation (in the order of JC) in a small region. Such contour points will be called turnpoints. </p><p>To indicate a line end, a turn must be of a limited length. Roughly speaking, the distance between the </p></li><li><p>436 ATTILA LERCH and DMITRY CHETVERIKOV: MACHINE-VISION SYSTEM </p><p>two polygon segments with opposite orientations should be close to the line width. Also, one can exploit here additional knowledge about the lines in the images considered: the lines are of relatively constant and uniform width. After detection of a pair of polygon segments that presumably contains a line end, a validity test is needed to check the directional consistency of the adjacent segments, since noise or inaccuracy in contour approximation can sometimes lead to false detection. Superfluous turnpoints may also appear when multiple points are detected at the same line end. Therefore, an algorithm for finding line terminations is proposed which exploits the approximately constant line width as a constraint and incorporates a procedure for validity checking of the found points. The main steps of the proposed algorithm are as follows: </p><p>Step I. Scan the contour polygon; for each vertex, hypothesize that the vertex is a turnpoint. Step 2. Test the hypothesis by a pairwise orien- tation comparison of the segment running into the examined vertex, with a sequence of segments following the vertex. Terminate Step 2 if a turn- point is found, or if the compared segments are too far from each other. Step 3. If a turnpoint is found, check the direc- tional consistency of the adjacent segments and discard superfluous points. </p><p>The framework of the algorithm is similar to those of corner detectors. Let us discuss Steps 2 and 3 in more detail. </p><p>A relation between the line width and the look-ahead distance limit has been derived for termination of Step 2. Compared to general corner detectors, this specific constraint speeds up the algorithm and considerably decreases the false detection rate. The relation formula contains a parameter that controls the sharpness of the turn to be detected. There is a tradeoff between the probability of detection and its spatial precision. More-reliable detection (which needs more steps to test the hypothesis) can be achieved at the cost of lower precision in the location of the detected line end. This can be tuned by the sharpness parameter. Practical considerations limit the range of the parameter and thus the execution time of the algorithm. </p><p>The basic constraint is as follows: </p><p>look_ahead_%stance&lt; line_widthlsir@, </p><p>where p is the angle (sharpness) parameter of the algorithm. Low values for /3 mean reliable detection but probably more computation and inaccurate location. The parameter 6 was set to approx n/4. </p><p>The line-end detection algorithm hypothesizes a vertex to be a turnpoint and examines the subsequent vertices until one of the following conditions is </p><p>satisfied: </p><p>-a turnpoint is detected; in date vertex must pass the ency test; </p><p>this case the candi- directional consist- </p><p>-the examined subsequent vertex is beyond the above look-ahead distance; then the hypothesis is rejected; </p><p>-scanning of the contour polygon is over; in this case the hypothesis is also rejected. </p><p>In Step 3, two procedures are used for filtering out the superfluous and false points. The first procedure discards a candidate turnpoint if the next vertex is also a candidate, since the next vertex lies closer to the actual line end. The second procedure applies back- ward and onward directional consistency tests. The backward test discards a candidate vertex if the seg- ment running into the vertex is short and the orien- tation of the segment preceding this segment is consi- derably different. The onward test is done in a similar way. The consistency tests make the algorithm less sensitive to small noisy peaks on line contours. It can be shown that a reasonable variation of the line width does not influence the performance of the algorithm. This value is used only as an upper estimate. This is import- ant in those applicat...</p></li></ul>