Deep Convolutional Neural Networks for surface quality inspection of hot long metal products

www.computervisionbytecnalia.com

Tecnalia Research & Innovation Computer Vision Group

Aitor Alvarez-Gila, Antonio Lopez-Cruz, Sergio Rodriguez-Vaamonde, Miguel Linares, Jose A. Gutierrez-Olabarria and Estibaliz Garrote

Conclusions and future work • Deep learning-based CNN-SURFIN classifier significantly outperforms all our

baselines supported by handcrafted features

• Deep learning-based end to end detection module is under development.

• Evaluation of alternative CNN architectures in progress, aiming at zero defects. Image database

• Custom image database collected from real production environment.

• Contains 3886 crops (256x256pixel) collected from long hot bars.

• Enables the evaluation of 2-class (OK/NOK) and 4-class classification tasks.

CNN-SURFIN We replaced the previous commercial software-based detection and classification module with an in-house made candidate window detection stage and a custom Convolutional Neural Network (CNN) performing the actual defect classification.

CNN-based classifier and training details:

• Custom architecture based on convolutional, ReLU and fully connected layers.

• Extensive data augmentation: illumination, scale, rotation, translation, focus, etc.

• Stochastic Gradient Descent with momentum.

• L2 and Droput -based regularization.

• Modified loss function to account for class imbalance.

Bibliography [1] D. Weimer, B. Scholz-Reiter, and M. Shpitalni, “Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection,” CIRP Annals - Manufacturing Technology.

[2] J. Masci, U. Meier, G. Fricout, and J. Schmidhuber, “Multi-scale pyramidal pooling network for generic steel defect classification,” in The 2013 International Joint Conference on Neural Networks (IJCNN), 2013, pp. 1–8.

[3] J. Masci, U. Meier, D. Ciresan, J. Schmidhuber, and G. Fricout, “Steel defect classification with Max-Pooling Convolutional Neural Networks,” in The 2012 International Joint Conference on Neural Networks (IJCNN), 2012, pp. 1–6.

[4] L. Yi, G. Li, and M. Jiang, “An End-to-End Steel Strip Surface Defects Recognition System Based on Convolutional Neural Networks,” steel research int., p. n/a-n/a, Apr. 2016.

[5] D. Soukup and R. Huber-Mörk, “Convolutional Neural Networks for Steel Surface Defect Detection from Photometric Stereo Images,” Advances in Visual Computing, pp. 668–677, Dec. 2014.

SURFIN Surface Quality Inspection System

Features:

• SURFIN performs real-time detection and classification of external defects in the continuous line manufacturing process of long metallic products.

• Works at early stages of the process, when the product is still incandescent (>1100ºC), preventing the unnecessary addition of value to it.

• Product types: bars, tubes, billets, slabs, beam blanks, structural profiles, etc.

• Defect types: roll marks, cracks, folds, scratch marks, holes, etc. (>0.3mm)

• Speeds up to 10m/s

• Portable (single camera) and cold-stage versions available

Architecture:

• Capture stage: co-aligned laser/LED illumination and hi-res line-scan camera sets (x3) held in the inspection portico (with integrated cooling).

• Detection stage: candidate window extraction

• Machine learning-based classification module

Portable SURFIN

Experiments and evaluation We evaluated our CNN-SURFIN architecture-based classifier in a 10-fold cross validation setup for two classification tasks. We compared it against the commercial SVM-based classifier, and implemented two additional baselines by extracting texture features (LBP) and training an SVM and a Random Forest classifier on top of these :

2-class classification (OK vs. NOK)

4-class classification (OK vs. NOK)

We retrained the same architecture for a 4-class problem, yielding an AUC of 0.9956.

Abstract

we present the advances incorporated into Tecnalia’s SURFIN surface quality inspection system. SURFIN performs real-time detection and classification of external defects from the manufacturing process of long hot (>1100ºC) metallic products, and has been extensively tested in real-life production environments. The system is based on laser/LED illumination and machine learning detection and defect classification techniques.

We upgraded SURFIN by adding an in-house made candidate window detection stage and a Convolutional Neural Network (CNN) for defect classification.

We validated the new classifier over a custom image database (with 2475 OK and 1411 defective hot tube images), finding that our deep learning-based approach significantly outperformed (AUC=0.9970) all our previous baselines (AUC=0.88-0.95).