ERCIM "ALAIN BENSOUSSAN" FELLOWSHIP PROGRAMME

Scientific Report

First name / Family name Younghak/ShinNationality Republic of KoreaName of the Host Organisation NTNUFirst Name / family nameof the Scientific Coordinator Ilangko BalasinghamPeriod of the fellowship 01/03/2016 to 28/02/2018

I – SCIENTIFIC ACTIVITY DURING YOUR FELLOWSHIP During the fellowship program, my main research topic was developing automatic polyp detection system during colonoscopy. Background and Goal of Study: Automatic polyp detection

systems having high precision and accuracy are necessary to help reducing the polyp miss-detection rates in colonoscopy diagnosis. We aim to design a whole framework of deep learning based automatic polyp detection systems that can detect polyp in colonoscopy images with accurate detection performance.

Materials and Methods: We apply a recent region-based convolutional neural network (R-CNN) [1] approach for the automatic detection of polyps in images obtained from colonoscopy examinations. We use a deep-CNN model (‘Inception Resnet’ [2]) as a pre-trained network and fine-tune the whole network using labeled polyp images. To overcome the small number of polyp images, we use image augmentation strategies for training the deep network. For evaluation, we use publicly available polyp-frame datasets which

were used in the recent challenge ‘Endoscopic Vision Challenge’ of MICCAI (Medical Image Computing and Computer Assisted Intervention) 2015 conference [3].

Results and Discussion: Performance of our deep learning based detection model is better than the results of other methods represented in Endoscopic Vision Challenge [3] in terms of all performance metrics: precision, recall, F1 and F2 scores (see below Table 1 for detail). Our model achieved a much larger True Positive, correctly detecting a total of 167 polyps out of a total 208 polyps in the test dataset, and with a smaller FP compared to all other teams. Furthermore, our model can detect difficult polyps (to see via the naked eye) as shown in below Figure 1.

TABLE 1. COMPARISON OF POLYP FRAME DETECTION RESULTS WITH OTHER STUDIES

Method True Positive

False Positive

False Negative

Precision (%)

Recall (%)

F1(%)

F2(%)

CUMED [3] 144 55 64 72.3 69.2 70.7 69.8

OUS [3] 131 57 77 69.7 63.0 66.1 64.2

UNS-UCLAN [3] 110 226 98 32.7 52.8 40.4 47.1

CUMED+OUS [3] 159 38 49 80.7 76.4 78.5 77.2

Our model 167 26 41 86.5 80.3 83.3 81.5

Figure 1. Detection examples of difficult polyps in test images. The first row shows the ground truth images of the test images below. The second row represents detection results from our model.

Conclusion(s): The deep learning based R-CNN method with the appropriate augmentation strategies is very promising for polyp detection tasks compared to other recent methods. We suggest two research directions based on current study. First topic is real-time polyp detection in colonoscopy by optimizing the deep networks or applying simple segmentation networks. Second one is polyp image synthesis for increasing different types of polyp training images that will be helpful for stable training of deep networks and improving

detection performance.[1] S. Ren, K. He, R. Girshick, and J. Sun, “R-CNN: Towards real-time object detection with region proposal networks,” in Advances in

Neural Information Processing Systems, Montreal, QC, pp. 91–99, 2015.[2] C. Szegedy, S. Ioffe, and V. Vanhoucke, “Inception-v4, inception-resnet and the impact of residual connections on learning”,

arXiv:1602.07261, 2016.[3] J. Bernal, N. Tajkbaksh,, F. J. Sánchez, J. Matuszewski, H. Chen, L. Yu, Q. Angermann, O. Romain, B. Rustad, I. Balasingham, K.

Pogorelov, S. Choi, Q. Debard, L. M. Hen, S. Speidel, D. Stoyanov, P. Brandao, H. Cordova, C. S. Montes, S. R. Gurudu, G. F. Esparrach, X. Dray, J. Liang and A. Histace, "Comparative Validation of Polyp Detection Methods in Video Colonoscopy: Results from the MICCAI 2015 Endoscopic Vision Challenge," IEEE Trans. Med. Imaging, vol. 36, no. 6, pp. 1231-49, 2017.

II – PUBLICATION(S) DURING YOUR FELLOWSHIP1. Younghak Shin and Ilangko Balasingham, “Comparison of hand-craft

feature based SVM and CNN based deep learning framework for automatic polyp classification”, 39th Annual International Conference of the IEEE Engineering in Medicine & Biology Society, Jeju Island, Korea, pp.3277-3280, 2017.

2. Younghak Shin, Heung-No Lee and Ilangko Balasingham, “Fast L1-based Sparse Representation of EEG for Motor Imagery Signal Classification”, 38th Annual International Conference of the IEEE Engineering in Medicine & Biology Society, Orlando, FL, USA, pp.223-226, 2016.

3. Nitin Rawat, Younghak Shin and Ilangko Balasingham, “EEG based image encryption via quantum walks”, 38th Annual International Conference of the IEEE Engineering in Medicine & Biology Society, Orlando, FL, USA, pp.231-234, 2016.

III – ATTENDED SEMINARS, WORKHOPS, CONFERENCESI have participated following three international conferences during the fellowship:1. 20th International Conference on Medical Image Computing and

Computer Assisted Intervention 2017, Quebec City, Quebec, Canada, September 10 to 14, 2017

2. 39th Annual International Conference of the IEEE Engineering in Medicine & Biology Society, Jeju Island, Republic of Korea, July 11 to 15, 2017.

3. 38th Annual International Conference of the IEEE Engineering in Medicine & Biology Society, Orlando, FL, USA, August 16 to 20, 2016.

IV – RESEARCH EXCHANGE PROGRAMME (REP)I did not participate the research exchange program during the fellowship period.