MORPHOLOGICAL FILTERS Wayne Weiyi Chen

1.1. INTRODUCTION Artificial intelligence is an attractive noun to every engineer because in some sense it represents a very well-developed technology such that we can implement the AI to let machine being able to mimic human being’s intelligence. Morphological filters in image processing is such a technology lies in the very fundamental layers of AI, it allows computer to simplify the information obtained so that we can use further programing to let the computers process the information such as pattern recognition.

In this problem, we will explore several morphological filters and implement them to two images for information digging.

In the first part of this problem, we will perform shrinking and thinning as well as some add-on programs related to morphological filters to counting the holes, single holes and pathways within the printed circuit board. In the second part, we will use shrinking and some other geometric programs to detect the missing teeth on a gear.

Figure 1. PCB.raw and GearTooth.raw

1.2. METHODOLOGY

1.2.1. Shrinking, Thinning and Skeletonizing In this problem, we will use a set of very powerful tools naming shrinking, thinning and skeletonizing. All of these three filters can be implemented using a form of 2-stage filtering. This is done by 2-stage instead of 1-stage due to the reason that we have to implement enormous amount of pattern comparison when using the latter so that the situation becomes extremely tedious and complicated.

For intuitive understanding, we can firstly give examples of shrinking, thinning and skeletonizing to illustrate their property and then introduce their detailed implementation. As shown in the Fig.13, we use three tools respectively to the white pixels of the gear tooth image.

Figure 2. Shrinking, Thinning and Skeletonizing the Gear

As we can see in the image, shrinking take the least care of the structure information of the original image and only give the topological information of the graph; thinning however give more information about the dimension of the graph, as we can see in the center part of Fig.13, the exist of teeth can be indicated in the thinned graph; skeletonizing, on the other hand, give the most structure information about the original graph, we can even tell the width of the gear tooth by measure the “Y” shape result.

There are more interesting properties lie in the different methods that we want to mention, however, we will keep them until the discussion part because they are related to our algorithms which are used to solve the problems.

1.2.2. The 2-stage Filters As mentioned above, we need to implement 2-stage filters in order to shrinking, thinning and skeletonizing the graphs in image.

Firstly we introduce the so called candidate selection filter. The candidate selection algorithm can be implemented by set up a hit and miss filter with different pattern tables named conditional mark patterns table. The reason of the taking this name is because after comparing with these patterns, we still need more extra condition to decide if we should keep the original value of that pixel unchanged.

To be specific, for each pixel in the image, if it is a target pixel (the pixel value is the one we want to shrink, thin or skeletonize), then we form its 8 neighbors, from its very right neighbor and anti-clockwise picking them, to a 8-bit character array, and send them to compare with the pattern table. For the purpose of efficiently speed up the program and reduce the complexity, we can firstly calculate the bond of these 8 neighbors with the central pixel. By doing this, we only need to compare the 8-bit character array with the patterns with the same bond number. Shrinking, thinning and skeletonizing have their own conditional mark patterns but the principle of this candidate selection is the same: if hit any pattern, we can mark the central pixel with “M”; if miss then we set its value to 0 in the M-map image. The M-map then is sent to stage-2 for further processing.

Stage-2 of the filter is called confirmation. Those pixels that are marked with “M” are sent to this stage for another round of pattern comparison. The output of pattern comparison in this stage is the so called “erasure inhibiting logical value”:

(5)

where if hit and if miss, is the 8 neighbors of the central pixel. The final output of this

filter can be written as

(6)

As can be clearly seen in this formula, we only erase the pixel value when and which means miss the pattern comparison.

Although the pattern comparison is similar with the one from stage 1, the process becomes trickier as there lays in much more patterns comparing to the stage 1. However, we can significantly reduce the complexity by some improvement applying to the simple 8-bit character comparison. This will be described in detail in the discussion part.

Another part need to be noted is that the process of shrinking, thinning and skeletonizing generally cannot be accomplished by one single usage of the two steps above. We need to design the program so that we can iteratively employ these two steps until the condition that M-map of the image is not changing any more.

( )0 1 2 3 4 5 6 7, , , , , , , ,P M M M M M M M M M

1P = 0P = { }iM

( ) ( )( )0 1 2 3 4 5 6 7, , , , , , , , ,G j k x M P M M M M M M M M M= Ç È

1M = 0P =

This can be done by setup a return value from the candidate selection: if there is no update in the M-map comparing to the last iteration, then return a mark such as zero value and therefore we can know when to jump out from the iteration, otherwise the iterative procedure will be continued.

1.2.3. Other Methods Using in the Solution Besides the shrinking, thinning and skeletonizing mentioned above, we have some other methods applied to the problem. They are both ad hoc method however efficiently solving the problem because they use the special properties of the graph after pre-processing. We leave the descriptions of these algorithms until the discussion part because we need to firstly have some analysis on the results from shrinking, thinning and skeletonizing.

1.3. RESULTS

1.3.1. Counting Holes and Pathways in the PCB The first problem is to count the holes in the PCB. The counting of the holes is complex if we directly develop an algorithm on search the original image; however, it is not difficult to observe that all of the holes can be shrunk into a single pixel. The counting of outlier points is much easier to implement. As a result, we need to firstly shrink the white pixels. We show the result in the Fig.14.

Figure 3. PCB.raw after Shrinking the White Pixels

As we expected, after the shrinking process, all of the holes become single white dots lie in a neighborhood filled with black pixels. We can now count the outlier white pixels to count the number of holes in the PCB: traverse the image, when there is a white pixel, check its 8 neighbors. If all of them are black pixels, then the count number plus 1.

Using this algorithm, we have the answer: there are 127 holes in total in the PCB.

The second part of this problem is to count the pathways in the PCB. This can be done by several methods, we will have a brief discuss about different methods in the discussion part and why we choose this particular one; however, here we want to only show all the results including the intermediate ones for a clear answer.

The first step is thinning process on black pixels, the result is in Fig.15.

Figure 4. The Black Pixels are Thinned in the PCB.raw

As we can see in this thinned graph, the PCB is composed by several pathways as well as bunch of single holes which cannot be counted as pathways. The challenge here is to use some algorithm to distinguish them and only count the pathways.

This can be done by using a simple but powerful tool which we give it an interesting and veritable name “infectious erasure”. The detailed algorithm will be discussed in the discussion part; here we only show the results of its output in the Fig.16.

The pathways are counted in such a way that we are able to distinguish the pathways and single holes so that the final result can be illustrated as two independent images at the same time we finish counting, one being the pathway-only image and the other being single-hole-only image.

Figure 5. Output Images of Counting Pathways:

(a) All of the Single Holes; (b) All of the Pathways

The process of the pathways counting is we traverse the image from the upper left corner to the lower right corner, the counts plus one if meets some specific criteria, and erase that pathway, continue the search. The final result of this process is shown in the Fig.16 (a). Fig.16 (b) showing all of the pathways can be obtained by “subtracting” the (a) from the original thinned graph in Fig.15.

As a final answer, there are 31 pathways in the PCB.

1.3.2. Locating the Missing Gear Tooth The final part of the problem is to investigate a gear with unknown missing teeth. What we do know is that there should be 12 teeth uniformly distributed around the gear. This information is crucial, because we are now able to calculate the location of the 12 teeth if we know the center of the gear and its radius.

The first step is to shrink the black pixels so that we can find out the locations of the centers for four black holes and therefore obtain the index of the gear center by taking the average of these four centers’. The shrunk graph is shown in Fig.17, note that we also show the reverse result (means black pixels are white and the white pixels are black in the real result after the shrinking) for the purpose of a better display effect.

The result of the shrinking is not only have four outlier black pixels indicating four centers but also includes two other single dots, one being on the left and the other laying at the lower left corner.

These two extra outliers can be eliminated by providing the extra location information of the gear, which is, telling the range where the gear lies in the image: We can provide the boundary information by giving the very left, right, up and low white pixels in the image. For example, if we want to know the most northern pixel of the gear, we can start searching the white pixel from the upper-left corner to the lower-right corner with column-priority order and return the first index set we hit; if we want to know the most eastern pixel of the gear, we can start searching from upper-right corner to lower-left corner with a row-priority order.

Figure 6. The Result of Shrinking the Black Pixels in GearTooth.raw:

(a) The Real Output Result; (b) the Reversed Output for the Purpose of Better Displaying

In the meantime of this procedure, we can obtain the outer radius of the gear by calculating the

distance between any of these four extreme points and the center of the gear.

Figure 7. An Illustration of the Algorithm

With showing Fig.18 we can have a clearer illustration of the algorithm: The black border marks out the region where the gear locates, we can search the outliers in this region to locate the center of the four black holes which also are marked out by four outlier black pixels in the Fig.18; we can also calculate the outer radius by the method mentioned above, the radius is marked by black lines in the image; and we can

know the 12 teeth’s location (marked by 25-point square in the Fig.18) by the equations:

gearR

gearR

( ),tooth toothi j

(7)

We can then check these locations : if there is white pixel then we have the tooth; if not, we

have the tooth missing. We show the final result in the Fig.19.

Figure 8. Missing Teeth: at angle and

1.4. DISCUSSION

1.4.1. Hit and Miss Filter in the Confirmation Step The pattern tables of stage 2 confirmation is more complicated than the ones from candidate selection and there are much more patterns that cannot be divided into groups by bond number. If we use the same method in the candidate selection, it will be a very time-consuming procedure.

Therefore, we introduce some tricks to avoid this and improve the performance of hit and miss filters in this stage.

• In the 8-bit character arrays, we mark all the “M”s as 1, “A, B and C”s as 2 and “D”s as 3.

• For each 8-bit character array we send for compare, we search all the patterns, and for each bit at any pattern:

o If that bit in the pattern is 0, then investigate corresponding bit at the 8-bit array, if it is not 0, then miss;

o If that bit in the pattern is 1, and the corresponding bit is not 1, then miss;

{ }

sincos

360 , 0,1,2,...,1112

tooth center gear

tooth center gear

i i Rj j R

n n

q

q

q

= -ìïí = +ïî

= Î!

"

"

"

( ){ },tooth toothi j

0 76p

o If that bit in the pattern is 2, check the corresponding bit, if it is 1, then mark a flag as 1;

o If that bit in the pattern is 3, then do nothing and continue.

o After all 8 bits are checked, if the situation “2” happened and the flag is still marked as 0, indicating none of the A, B and C is 1, then miss.

This method avoids us from writing out all of the patterns in the table and also speeds up the program because we need not to compare all of the patterns in the table.

1.4.2. Detail and Rationale of Algorithms when Counts Pathways As stated in the previous section, to count the pathways is not as easy as counting the holes in the PCB. This is because in the latter case, we only need to shrink the white pixels until it converges (M-map does not change any more) and count the white outliers in the image. In the pathway case, there are many patterns we need to eliminate, including single holes.

An intuitive solution to this problem at first is to locate the holes and to fill the holes with black and then shrink the black pixels. The result of that would be similar to the first part of this problem: we need to calculate the black outliers to count the pathways. However, this is difficult to implement because we need extra information for the holes to fill them with black pixels; also this procedure will be very time-consuming because there are at least two sets of shrinking iterations.

As a result, we develop a much easier and simpler method to count the pathway. As a fact, this method only need 8 iterations to total convergence when thins the black pixels and then only a one-time traversing the whole image to eliminate all of the single holes and count the pathway simultaneously.

An Important Property of Pathways after Thinning As shown in the Fig.20, there is a common property share by all the pathways (examples shown in the first 5 of Fig.20) after thinning that distinguished them from single holes (right-bottom of Fig.20): there is at least one pixel in a pathway that has more than 2 neighbors in the black color. All of the pixels from single holes have 2 neighbors in the black color. Interestingly, this property does not exist when we using shrinking as shown in the Fig.21. This is due to the reason that after shrinking, pixels those who construct the two edges of right angles in the single holes have more than 2 neighbors being in the black color --- this is exactly the same condition that we use to distinguish the pathways from single holes.

Figure 9. Pathways vs. Single Holes

Figure 10. Single Holes after Shrinking; the Reason Why NOT Using Shrinking:

After shrinking, there are right angles in the circles shrunk from single holes (like upper-left corner of both of these two examples), resulting in pixels that have more than 2 nighbors being in black color

“Infectious Erasure” Known this important property that distinguishes the pathways from single holes, we need to develop an effective method to count the pathways. The algorithm we use to count can be described as below:

• According to the property we find above, search black pixels that have more than 2 neighbors in the same color from the upper-left corner to the lower-right corner.

• If there is a hit, then the count number plus 1, at the same time erase that pathway.

Here we use the so called “infectious erasure” to eliminate the pathway we have already taken into count. This algorithm can be implemented using a recursive function:

• Start with the pixel where there is a hit mentioned above.

• Immediately set that pixel to white color. Search its neighbors, those who are black pixels are recursively sent to the same function where set them to white color and further search their neighbors.

• The recursion ends at the pixels that have no more black neighbors.

The name “infectious erasure” is because we delete the black pixels one by one as they are infected by their neighbors. The final result of this procedure is shown in the Fig.16 (a). Here we show some intermediate results of this algorithm for a better illustration.

Figure 11. Examples: Counting Pathways at Step 1, 9, 17 and 24

1.4.3. Performance Regarding to Shrinking, Thinning and Skeletonizing There may be concerns that why our result in Fig.17 contains extra points besides the four expected center point. This can be answered by investigating a minor step before we perform shrinking (the same with thinning and skeletonizing).

We have to perform border expansion because the morphological process undergoes within windows. In the gear tooth case, we add a one-line black border to the original image. It seems that the shrinking breaks down the topology property of the black background: the background is surrounding the gear so that the result should be a circle instead of the extra points shown in the Fig.17.

A closer investigation can explain this. If we set the added border to be white, we can have the result below (the color is reversed for the purpose of better displaying):

The background (now in white) is now a circle and is not broke down to points, and our shrinking algorithm is not going wrong.

Another interesting result from exploring this is we can sometimes speed up the convergence by adding an opposite color as a border expansion: it takes 95 iterations to converge when we get Fig.17 while the figure above is a converged result of only 48 iterations. This is because the shrinking can be performed simultaneously from both inside from the border of the gear and outside from our black border. However, we obtain the same information for our interest by both of these two processes. So we can say that to add a black background is more efficient.

Finally we want to point out that we should combine the three morphological algorithms with other geometrical methods (like we use in the gear tooth one) as well as other algorithms (like the one we use to counting the pathway) because shrinking, thinning and skeletonizing are relatively complicated and time-consuming processes, though by using them we can obtain crucial information.