Zhao Liu, Josiane Zerubia

Ayin Research Team, INRIA Sophia Antipolis Méditerranée (France)

Publication available on https://team.inria.fr/ayin/publications-hal/

Illumination Modeling and Chromophore Identification in

Dermatological Photographs for Skin Disease Analysis

Ayin

2

INTRODUCTION

Objectives:

Imaging artifact removal from skin images

Chromophore concentration identification

Applications:

Computer-aided melanoma diagnosis

Automatic acne detection and differentiation

3

SKIN IMAGES

Fig. 1 Skin lesion images from a free public database [DermQuest, 2013] with illumination artifacts. (a) A superficial

spreading melanoma image with obvious horizontal shading effects. (b) Another superficial spreading melanoma

image with palpable highlight effects within the lesion areas.

Visual perception of skin color is not only credited to major

chromophores (melanin and hemoglobin) underneath the skin surface,

but is also affected by external illumination and spectral responses of

imaging detectors.

Pigmentation information may partially concealed by imaging artifacts.

4

OPTICS OF HUMAN SKIN

Fig.2 A three-layered skin reflectance model.

5

OPTICS OF HUMAN SKIN (Cont.)

Diffuse reflectance only:

where Qλ stands for sensor characteristics, kd is diffuse reflection constant, Ed,λ is intensity of diffuse components of light source. wd is wavelength-independent shading variation due to scene geometry. μm,λ and μh,λ are wavelength-dependent absorptive coefficients of melanin and hemoglobin, respectively.𝑙m,λ and 𝑙h,λ are the lengths of light penetration in epidermis and dermis

layers. cm and ch are densities of melanin and hemoglobin in a sampled volume of skin.

• Previous studies [Claridge, 2003; Yamamoto, 2008; Madooei, 2012] consider

human skin as a diffuse reflectance model only.

• No specular reflectance on the skin surface.

• Shading effects can be removed by dividing two spectral bands.

6

OPTICS OF HUMAN SKIN (Cont.)

Specular and diffuse reflectance:

where ks is specular reflection constant, Es,λ is intensity of specular components of light source.

ws is the specular factor giving rise of highlight, and α is a material relevant constant.

• Joint effects of specular and diffuse reflectance.

• Address artifacts (e.g. highlight, shading) normally existing in large-

dynamic-range intensity skin images captured under uncontrolled

environment.

7

METHOD

Specular Reflection and Diffuse Reflection Separation

Decomposition of Diffuse Reflection Image

Melanin Index and Hemoglobin Index Estimation

8

Estimates the maximum diffuse image [Yang, 2010] and apply it as intensity

guidance in a weighted average process to remove specular reflection.

Surface texture will be filtered out as specular component and resultant

diffuse skin image turns out blurred.

Localize the candidate specular reflection areas and then decide the highlight

regions by selecting pixel intensity larger than a threshold.

To avoid the problem of fake intensity (black spots) values, the diffuse

chromaticity of the detected specular pixels is computed by a B-spline image

interpolation [Pan, 2003].

Specular Reflection and Diffuse Reflection Separation

9

Specular Reflection and Diffuse Reflection Separation (Cont.)

Fig. 3 Specular reflection removal in skin color image. (a) The original melanoma images having highlight and

shading as artifacts in Fig.1. (b) Candidate specular reflectance. (c) Blurred diffuse reflectance image by

subtracting specular reflection (b) from (a), pointing out the areas having fake intensity pixels by ellipses. (d)

Specular reflectance component after thresholding. (e) Diffuse reflectance image using a B-spline pixel

interpolation calculated by our method.

(a)

(b) (c)

(d) (e)

10

Decomposition of Diffuse Reflection Image

Diffuse image intensity can be formulated by a linear combination of

chromophore coefficients, optical parameters of light source, and effects of

scene geometry in an inverse logarithmic form:

Applied an adaptive bilateral filter to the inverse logarithmic diffuse skin

image, where image intensity gradient is applied as reference to make the

spatial and range standard deviations in Gaussian kernel adaptive to each

pixel.

Considering that shading is normally a low frequency component, the high

frequency chromophore elements can be smooth out through embedding the

bilateral filtering into an iterative process.

11

Decomposition of Diffuse Reflection Image (Cont.)

Varied spatial and range standard deviations at different image location

make the illumination estimation unnatural and less homogeneous.

A weighted parametric polynomial curve fitting is subsequently introduced

to generate the final illumination image.

Subtracting the illumination map from the diffuse reflection component, a

skin image can be decomposed into a base layer and a detail layer.

12

Fig. 4 Base layers and detail layers of the diffuse melanoma image in Fig. 3(e). (a) Derived illumination component. (b) Corrected image after

specular reflection and shading effect removal by the proposed method. (c) Spectral bands of diffuse reflection image. (d) Intensity gradient

expressed by a monotonically increasing function. Blue color refers to small value, while red color stands for large one. (e) Initial illumination

approximation by the adaptive bilateral filtering. (f) Base layers (𝐼𝑏𝑎𝑠𝑒) by the polynomial curve fitting, taking (e) as a prior. (g) Detail layers

𝐼𝑑𝑒𝑡𝑎𝑖𝑙 after subtraction. For comparison with the original spectral band in (c), (e), (f) and (g) are shown in the exponential form. 1st – 3rd columns:

Red, Green and, Blue channels, respectively.

13

Melanin Index and Hemoglobin Index Estimation

Fig. 5 Spectral absorption of major

chromophores from 400nm to 1,000nm based

on published data [Jacques, 1998], and their

relation with spectral responses of conventional

RGB digital cameras.

From Fig. 6, the diffuse reflectance of

skin in a specific spectral band can

then be considered as an effect

attributed to particular chromophores.

Taking the absorptive coefficients and

light penetration lengths from the

previous publications [Keller, 2001,

Jacques, 1998], the melanin density

and hemoglobin density can be

estimated.

14

Melanin Index and Hemoglobin Index Estimation (Cont.)

Fig. 6 Melanin and hemoglobin concentrations calculated by different algorithms. (a) Melanin index and (b)

hemoglobin index derived from the proposed approach. (c) First and (d) second independent components by

the ICA method [Madooei, 2012].

15

EXPERIMENTS AND RESULTS

Melanoma Diagnosis

Lesion segmentation after illumination correction.

Melanoma classification using chromophore descriptors.

Acne Detection and Differentiation

Semi-automatic acne detection.

Inflammatory acne and hyperpigmentation discrimination.

16

Melanoma Diagnosis

Experimental dataset

A number of 201 conventional RGB skin lesion images (62MMs, 139 benign nevi (BN)) are collected from free public databases [DermQuest, 2013; DermIS, 2013] to form an experimental dataset.

Of these lesions, 135 were reported to be excised and examined by histopathology, giving 62 MMs and 73 BN diagnosed as 34 Dysplastic nevi, 25 Common Acquired nevi1, 8 Blue nevi, 4 Spitz nevi, and 2 Seborrheic Keratosis.

The remaining 66 lesions did not undertake excision biopsy due to no evidence of malignancy under clinical examinations.

17

Lesion Segmentation

The proposed illumination modeling method are compared with two other

illumination correction approaches [Cavalcanti, 2011, Glaister, 2013], both

of which were developed specifically for skin lesion images.

The manual segmentation outlined by an experienced dermatologist is

applied as the reference truth.

Skin images after illumination modeling are segmented into lesion and

non-lesion areas by the Otsu’s method [Otsu, 1979].

18

Fig. 7 Comparison of different illumination modeling methods for skin lesion images. (1st row) Six example lesion images from free public

databases [DermQuest, 2013, DermIS, 2013] with highlight and shading as artifacts. (2nd row) Results from Cavalcanti’s method [Cavalcanti,

2011]. (3rd row) Results from MSIM method [Glaister, 2013]. (4th row) Results by the proposed method.

Illumination Modeling Comparison

19

Segmentation Comparison

Fig. 8 Skin lesion segmentation on images after illumination correction by different methods in Fig. 7.

Black line: manual segmentation by an experienced dermatologist. Blue line: segmentation on the

original image. Green line: segmentation on the corrected image by the Cavalcanti’s method. Yellow

line: segmentation on the corrected image by the MSIM method. Red line: segmentation on the

corrected image by the proposed method.

20

Lesion Segmentation (Cont.)

Average TC ± Standard Deviation (%)

Original 65.74 ± 21.87

Cavalcanti [Cavalcanti, 2011] 79.40 ± 11.73

MSIM [Glaister, 2013] 81.06 ± 13.92

Our method 88.47 ± 8.16

Table 1. Comparison of segmentation accuracy on skin lesion images after illumination

correction using different algorithms.

The Tanimoto coefficient (TC) [Tan, 2005] defined below is used to evaluate

the automatic segmentation accuracy.

21

Melanoma Classification

SE (%) SP (%) Acc. (%) AUC

RGB Original 66.13 69.78 68.65 0.653

RGB Cavalcanti [Cavalcanti, 2011] 70.97 77.70 75.62 0.713

RGB MSIM [Glaister, 2013] 70.97 81.30 78.11 0.752

RGB, Our method 72.58 88.50 83.58 0.852

MI+HI, ICA [Madooei, 2012] 77.42 86.33 83.58 0.866

MI+HI, Our method 83.87 84.89 84. 57 0.878

Table 2. Melanoma diagnostic results applying diagnostic features derived from RGB colorimetry,

MI+HI from the ICA method [Madooei, 2012], and MI+HI from the proposed method.

A 12D diagnostic feature vector, including 3 absolute color variation measures, 3

relative color variation measures, and 6 pigmentation asymmetry measures, is applied

as the diagnostic features for melanoma classification.

A linear support vector machine is used as the classifier, and a 10-fold cross validation

is the training-testing strategy.

Sensitivity (SE), specificity (SP) and overall diagnostic accuracy (Acc.) are recorded to

evaluate classification performance.

22

Acne Detection and Differentiation

Acne vulgaris is believed resulting from proliferation of propionibacterium

acnes in pilosebaceous unit [Webster, 1995].

Inflammatory acne, visible as red papules, is due to increase of

oxyhemoglobin level.

Dark scar, known as secondary acne hyperpigmentation, will appear

thereafter thanks to excess melanin production during the healing process

[Webster, 1995].

23

Acne Detection and Differentiation (Cont.)

The combination of melanin and hemoglobin indices CI is used as the only

pigmentation descriptor and forwarded to a MRF model [Liu, 2013] to

isolate acne from surrounding normal skin.

A number of 50 challenging RGB acne images, captured under uncontrolled

environment from a free public database [Dermnet NZ, 2013], have been

tested.

Professional assessment of acne images is under process by an experienced

dermatologist in the Hôpital L’Archet 2, CHU de Nice.

24

Fig. 9 Acne segmentation of two conventional RGB images using chromophore descriptors derived from different methods. (a)(b) Original

images with highlight/shading. (c)(e) Results by the bilateral decomposition. (d)(f) Results by the ICA method [Madooei, 2012].

25

Conclusions

Proposes a novel illumination modeling and chromophore identification

method in dermatological photographs for skin disease analysis.

The derived melanin and hemoglobin indexes are well identified, and prove

robust to highlight and strong shading in skin color images captured under

uncontrolled imaging setting.

Experiments show that the proposed method increases automatic melanoma

diagnostis, and exhibits a high potential for other skin disease analysis.

For the next step study, the accuracy of automatic acne segmentation will be

compared to the manual segmentation outlined by the experienced

dermatologist when the assessment is available.

26

References

[DermQuest, 2013] DermQuest. Sept. 23th 2013. [Online] Available: http://www.dermquest.com.

[Claridge, 2003] E. Claridge, S. Cotton, P. Hall, M. Moncrieff. “From color to tissue histology: Physics-based

interpretation of images of pigmented skin lesions,” Med. Image Anal., vol. 7, pp. 489–502, 2003.

[Yamamoto, 2008] T. Yamamoto, H. Takiwaki, H. Ohshima. “Derivation and clinical application of special

imaging by means of digital cameras and image freeware for quantification of erythema and pigmentation,” Skin

Res. Technol., vol. 14, pp. 26–34, 2008.

[Madooei, 2012] A. Madooei, M. Drew, M. Sadeghi. M. Atkins. “Intrinsic Melanin and Hemoglobin Color

Components for Skin Lesion Malignancy Detection,” In proc. of 15th Medical Image Computing and Computer

Intervention (MICCAI), vol. 7510, pp. 315-322, Nice, 2012.

[Yang, 2010] Q. Yang, S. Wang, N. Ahuja. “Real-time Specular Highlight Removal Using Bilateral Filtering,” In

proc. of 11th European Conference on Computer Vision (ECCV), vol. IV, pp. 87-100, Greece, 2010.

[Pan, 2003] J. Pan. “Image Interpolation using Spline Curves,” Dept. of Mechanical Engineering MEC572 term

paper, 2003.

[Jacques, 1998] S. Jacques. “Skin optics,” Oregon Medical Laser Center News, January, 1998.

[Keller, 2001] G. Keller, V. Lacombe, P. Lee, J. Watson. Lasers in Aesthetic Surgery, Thieme, 2001.

27

References

[DermIS, 2013] DermIS. Sept. 23th 2013. [Online] Available: www.dermis.net.

[Cavalcanti, 2011] P. G. Cavalcanti and J. Scharcanski, “Automated prescreening of pigmented skin lesions using

standard cameras,” Comput.Med. Imag. Graph., vol. 35, pp. 481–491, 2011.

[Glaister, 2013] J. Glaister, R. Amelard, A. Wong, and D. A. Clausi. "MSIM: Multistage Illumination Modeling of

Dermatological Photographs for Illumination-Corrected Skin Lesion Analysis," IEEE Trans. Biomed. Eng., vol.60,

no.7, pp.1873-1883, 2013.

[Otsu, 1979] N. Otsu. "A threshold selection method from gray-level histograms". IEEE Trans. Sys., Man., Cyber.

Vol.9, no.1, pp.62–66, 1979.

[Tan, 2005] P. Tan, M. Steinbach, V. Kumar. Introduction to data mining, 1st edition, Boston, Pearson Addison

Wesley, 2005.

[Webster, 1995] G. Webster. “Inflammation of acne vulgaris,” J. Am. Acad. Dermatol., vol. 33, pp. 247–253,

1995.

[Liu, 2013] Z. Liu, J. Zerubia. “Towards automatic acne detection using a MRF model with chromophore

descriptors.” The European Signal Processing Conference (EUSIPCO), in press, Marrakech, 2013.

[Dermnet NZ, 2013] Dermnet NZ. Oct. 22nd 2013. [Online] Available: http://www.dermnetnz.org/.