AUTOMATIC COLOR CLASSIFICATION

METHOD IN HUMAN TEETH USING MACHINE

LEARNING

Advisor: MSc. Roger Guzmán

Members:

Arnold Herrera. Cod:625569

Cristhian Arce. Cod:625577

Outline

• Problem statement

• Research question

• Objectives

• Workflow Methodology

• Methodology development

• Results

• Conclusions

• Future Works

• References

Problem Statement (1/2)

Figure 1. Tooth lost.

Te

eth

lost

Age

Figure 2. Partial prostheses. Figure 3. Acrylic Teeth for

prostheses fabrication. Figure 4. Vita® Guide.

Problem Statement (2/2)

Figure 5. Subjectivity. Figure 6. Time lost. Figure 7. Incompatibilities

Research Question

How to perform the process of color classification in teeth taking as reference

the VITA color guide, to support the process of color identification in human

teeth using artificial intelligence?

Figure 8. Research Question

To implement an automatic method using machine learning for the color classification in 2D images of

human teeth.

ObjectivesGeneral Objective

Specifics Objectives

● To build a dataset from photographic images of human teeth, to perform the classification of tooth

color.

● To design a classification strategy using machine learning techniques for dental color classification.

● To implement an algorithm based on machine learning for the classification of color in human teeth.

● To evaluate the accuracy of the method implemented using techniques to measure performance

such as F- Beta Score and confusion matrix.

Workflow Methodology

Figure 9. Workflow

1 3

2 4

5

6

7

DATASET

PREPROCESSING

FEATURE EXTRACTION CLASSIFICATION

SAMPLING RESULTS

PERFORMANCE MEASURES

Methodology Development(Dataset Building)

PhotographyResize and manual

segmentation Labeling

Figure 10. Dataset Building

Methodology Development(Workflow Preprocessing)

Image 11: Workflow preprocessing

1 3

2 4

5

6ORIGINAL

IMAGE

HSV CHANNELSOF IMAGE

HISTOGRAM OF HUE CHANNEL FROM

THE IMAGE

OVERLAY MASK TO THE IMAGE

MASK FROM HSVCHANNEL VALUES

FIND BIGGESTCONTOUR IN THE IMAGE

Methodology development

Figure 12. Get

value of moments

Feature Extraction (1/2)

(Color Moments)• First Moment: Mean

𝑥 = 𝐸𝑥 = 𝑖=1

1

𝑁

The first moment is the color average present in the image.

• Second Moment: Variance

𝜎2 =1

𝑁| 𝑥𝑖 − 𝑥 2|

In images the second moment measure the dispersion of the data in a sample

respect to the mean.

• Third Moment: Skewness

𝑆 =1

𝑁| 𝑥𝑖 − 𝑥 3|

The third moment is the grade of skewness of the color distribution on the image

Feature Extraction (2/2)

(Color Moments)

Methodology development

Moment Generating Function

𝑚𝑛 = 𝑖=1

1

𝑁 𝑥𝑖 − 𝐸𝑥

𝑛

• 𝑚𝑛 = The 𝑛𝑡ℎmoment of the distribution

• N = Total of elements in the array

• 𝑥𝑖 = The value of the 𝑖𝑡ℎpixel

• E[x] = The expected value of the distribution

• n = Represent the number of the moment

Sampling(Cross Validation K – Folds)

Image 13 Sampling

Dataset

Test

20%

Training

80%

Test

25%

Training

75%

Validation Train Train Train Train

Validation

Train

Train

Train

Train

Train

Train

Train Train

Train

Validation

Train

Train

Validation

Train

Train

Validation

Train

Train

Train

Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration 5

Validation

Score#1Validation

Score#1

Validation

Score#1

Validation

Score#1

Validation

Score#1

Classification Algorithms(1/3)Support Vector Machines (SVM)

Vector support machines is a learning algorithm that uses statistical

regression to define so-called straight lines if they are in two

dimensions, flat if they are in three dimensions or hyperplanes.

H1: 𝑤 ∙ 𝑥𝑖 + 𝑏 ≥ 1 𝑤ℎ𝑒𝑛 𝑦𝑖 = +1H2: 𝑤 ∙ 𝑥𝑖 + 𝑏 ≤ 1 𝑤ℎ𝑒𝑛 𝑦𝑖 = −1

The distance between the dividing hyperplane to the best margin is:

𝐵𝑒𝑠𝑡 𝑀𝑎𝑟𝑔𝑖𝑛:𝑤 ∙ 𝑥𝑖 + 𝑏

𝑤=

1

𝑤

Image 14: Support Vector Machine

H1

H2

Classification Algorithms (2/3)Decision Tree (DT)

Decision Trees (DTs) are a non-parametric

supervised learning method used for

classification and regression. Decision trees

learn from data to approximate a sine curve

with a set of if-then-else decision rules. The

deeper the tree, the more complex the

decision rules and the fitter the model.

Image 15 Decision Tree

Inputs = X1 Inputs = X2

Classification Algorithms (3/3)K – Nearest Neighbors (KNN)

K-Nearest Neighbours is one of the most basic yet essential

classification algorithms in Machine Learning. It belongs to

the supervised learning domain and finds intense application

in pattern recognition, data mining and intrusion detection.

This algorithm use some distances to compute which data

belong to a class.

• Euclidean: 𝑖=1𝑘 𝑥𝑖 − 𝑦𝑖

2

Image 16: K – Nearest Neighbors

Results (1/5)(Data Set)

Image 19 Dataset Source: Authors.

0

100

200

300

400

500

A1 A2 A3 A3.5

242

461

263

63

Dataset Population

Results (2/5)

(Data Set)

Source: Authors.Image 21 XML CoordinatesImage 20 Labels

Results (3/5)(Preprocessing and feature extraction)

Image 24: Numerical representation of the imageSource: Authors.

Image 22: Color histogramImage 23 Interest Region

Results (4/5)(Classification Methods)

Source: Authors.Table 1: Performance measures results

Precision Recall F1-score Precision Recall F1-score Precision Recall F1-score

A1 0,184444 0,02235 0,038364 0,212322 0,203164 0,190052 0,29167 0,366382 0,305156

A2 0,450816 0,985498 0,615032 0,454616 0,672798 0,511238 0,453752 0,541002 0,487752

A3 0 0 0 0,174142 0,206302 0,175258 0,265664 0,162428 0,187388

A35 0 0 0 0,080804 0,088884 0,08444 0 0 0

Accuracy 0,447302 0,364842 0,36

Macro avg 0,158814 0,251964 0,163346 0,225472 0,284454 0,235974 0,25277 0,267454 0,245076

Weight avg 0,24147 0,447302 0,288498 0,299136 0,376664 0,313176 0,360182 0,364736 0,347572

Support Vector Machine K - Nearest NeighborDecision Tree

Results (5/5)(Proposed Classification Strategy)

Source: Authors.

1 3

2 4

COLOR BASED SEGMENTATION

COLOR MOMENTS IN RGB

SUPPORT VECTOR MACHINE

RESULTS

Conclusions

● This experiment discusses the accuracy of various classification methods that include support

vector machines, decision trees and K – nearest neighbors, in order to classify the color of teeth

in humans, resulting in the support vector machines are the most accurate when identifying the

color of a tooth.

● A 45% accuracy result was obtained taking into account that it is the first investigation that seeks

the classification of tooth color in non-clinical images

● Decision trees have a better representation of the classes that are least represented, compared

to the other two classification methods used.

● Was constructed the first dataset of non-clinical images of teeth by classifying every image

according to the VITA® color guide.

Future Works

● It is recommended to increase the dataset samples according to a strategy that allow guarantee a

balanced dataset for improve the performance of the classification strategy.

● It is recommended the implementation of an automatic method for smile detection, this will help in

the preprocessing stage reducing all this to a single step omitting the actual preprocessing

strategy.

● It is recommended to explore some other representative color features, to compare the results

with the current experiment.

● Once finished it would be good to start some other research for teeth size classification to solve

the whole problem of conversion between acrylic teeth brands.

References (1/5)

● Abraham, Celeste M., ‘A Brief Historical Perspective on Dental Implants,

Their Surface Coatings and Treatments’, The Open Dentistry Journal, 8.1

(2014), 50–55 <https://doi.org/10.2174/1874210601408010050>

● Albalat, Javier, ‘La Importancia de La Estética Dental’, 2016, p. 1

<http://albalatdental.com/la-importancia-de-la-estetica-dental/> [accessed

7 February 2019]

● Allibha, Eijaz, ‘Holdout vs. Cross-Validation in Machine Learning – Eijaz

Allibhai – Medium’ <https://medium.com/@eijaz/holdout-vs-cross-

validation-in-machine-learning-7637112d3f8f> [accessed 3 March 2019]

● Baxter, R., N. Hastings, A. Law, and E. J.. Glass, A Study of Dental Color

Matching Color Selection and Color Reproduction, Animal Genetics, 2008,

xxxix <http://www3.unisi.it/dl2/20100304131719197/Corciolani.pdf>

References (2/5)

● Begoña, Tortolero María, ‘La Autoestima En Niños y Adolescentes Con

Alteraciones Dentarias. Revisión Bibliográfica Self-Esteem in Children and

Teenagers with Tooth Disorders. Literature Review Resumen’, 16.2

(2015), 58–70 <http://servicio.bc.uc.edu.ve/odontologia/revista/vol16-

n2/art06.pdf>

● Clara, Santa, Villa Clara, Santa Clara, and Villa Clara, ‘La Odontología

Estética Como Arte The Aesthetic Dentistry as Art’, 8.4 (2014), 107–9

● Color, A, ‘Anexo A . Color y Transformaciones’

● Deb, Sagarmay, ‘Multimedia Systems and Content-Based Image

Retriveal’, 2004, 15–34

● Giraldo C, Gomez L, ‘Percepción de La Estética Dental En Adultos de

Nivel Socioeconómico Medio Alto En Bogotá: El Primer Paso Para

Proponer Tácticas de Comunicación Para El Paciente a Una Compañía

Multinacional de Ortodoncia’, Statewide Agricultural Land Use Baseline

2015, 1 (2015), 50–55 <https://doi.org/10.1017/CBO9781107415324.004>

References (3/5)

• González, William Almonacid, ‘Colombia: El Paradigma Existencial de La Violencia’,

Pensamiento, Palabra y Obra, 17, 2017, 68–77

<http://www.scielo.org.co/pdf/ppo/n17/n17a06.pdf>

• Griego, Miguel Romero, ‘La Estética En La Vida Humana’, Revista Académica de

La ENP3, 2015, 1–3

• Haddad, Helene J., Ziad Salameh, Walid Sadig, Moustafa Aboushelib, and Holger

A. Jakstat, ‘Allocation of Color Space for Different Age Groups Using Three-

Dimensional Shade Guide Systems.’, The European Journal of Esthetic Dentistry :

Official Journal of the European Academy of Esthetic Dentistry, 6.1 (2011), 94–102

• International, Commission, and The Cie, ‘Color Models CIE Space for Color

Matching CIE 1931 Model’, Color Models CIE Space for Color Matching CIE 1931

Model, 2006

• Isistan, Silvia Schiaffino, ‘Inteligencia Artificial Técnicas de Clasificación’, 2018

• Keen, Noah, ‘Color Moments’, School Of Informatics, University Of Edinburgh,

2005, 3–6

References (4/5)

• López Boada, Jesús, Beatriz López Boada, and Vicente Días López, ALGORITMO

DE APRENDIZAJE POR REFUERZO CONTINUO PARA EL CONTROL DE UN

SISTEMA DE SUSPENSIÓN SEMI-ACTIVA (Madrid, 2004)

<https://www2.uned.es/ribim/volumenes/Vol9N2Julio_2005/V9N2A08 Lopez Boada.pdf> [accessed 18 February 2019]

• Manament Solutions, ‘Machine Learning Una Pieza Clave En La Transformación de

Los Modelos de Negocio’, 2018

<https://www.managementsolutions.com/sites/default/files/publicaciones/esp/machi

ne-learning.pdf>

• MHT Optic Research AG, ‘MHT Optic Research AG’, p. 16

<http://www.mht.ch/index.php?uri=micro.html> [accessed 1 March 2019]

• Molinero Díez, Gregorio, ‘3. Espacios de Color’, 5–10

<http://bibing.us.es/proyectos/abreproy/11875/fichero/Proyecto+Fin+de+Carrera%2

52F3.Espacios+de+color.pdf>

• Noble, William S, What Is a Support Vector Machine?, NATURE

BIOTECHNOLOGY, 2006, xxiv <http://www.nature.com/naturebiotechnology>

[accessed 18 February 2019]

References (5/5)

• Odontologia, Facultad D E, ‘Guia de Atención En Rehabilitación Oral’, 2013, 50

• Sánchez Giménez, Fulgencio, Manual Básico Del Tratamiento Protésico Para

Odontólogos, 2016

• Schmeling DDS, MS, PhD, Max, ‘Color Selection and Reproduction in Dentistry.

Part 3: Visual and Instrumental Shade Matching’, Odovtos - International Journal of

Dental Sciences, 19.1 (2017), 23 <https://doi.org/10.15517/ijds.v0i0.28083>

• Sîmpălean, Dan, Marius D Petrişor, Marius Ş Măruşteri, Vladimir Bacârea, Tudor

Călinici, and Simona Băţagă, ‘A Software Application to Detect Dental Color’, 37.3

(2015), 31–38 <2018-11-26> [accessed 26 November 2018]

• Soto, Araya Constanza Andrea, ‘EVALUACIÓN DEL IMPACTO PSICOSOCIAL Y

AUTOPERCEPCIÓN DE LA ESTÉTICA EN PACIENTES SOMETIDOS A

BLANQUEAMIENTO DENTAL EXTRACORONARIO’ (UNIVERSIDAD DE CHILE,

2017)

• Vanini, L., and F. M. Mangani, ‘Determination and Communication of Color Using

the Five Color Dimensions of Teeth.’, Practical Procedures & Aesthetic Dentistry :

PPAD, 13.1 (2001)