LEAF BOUNDARY EXTRACTION AND GEOMETRIC MODELING OF VEGETABLE SEEDLINGS Ta-Te Lin, Yud-Tse Chi,...

LEAF BOUNDARY EXTRACTION AND GEOMETRICLEAF BOUNDARY EXTRACTION AND GEOMETRICMODELING OF VEGETABLE SEEDLINGSMODELING OF VEGETABLE SEEDLINGS

Ta-Te Lin, Yud-Tse Chi, Wen-Chi LiaoTa-Te Lin, Yud-Tse Chi, Wen-Chi Liao

Department of Bio-Industrial Mechatronics Engineering,Department of Bio-Industrial Mechatronics Engineering,National Taiwan University,National Taiwan University,

Taipei, Taiwan, ROCTaipei, Taiwan, ROC

INTRODUCTIONINTRODUCTION

Plant growth measurement and Plant growth measurement and modelingmodeling

Image processing techniqueImage processing technique Seedling characteristicsSeedling characteristics ApplicationsApplications

OBJECTIVESOBJECTIVES

To develop image processing algorithms for leaf To develop image processing algorithms for leaf boundary extraction.boundary extraction.

To model leaf boundary with Bezier curves and To model leaf boundary with Bezier curves and develop leaf features based on Bezier curve.develop leaf features based on Bezier curve.

To determined leaf features of selected vegetable To determined leaf features of selected vegetable seedlings based on basic morphological descriptors, seedlings based on basic morphological descriptors, Fourier descriptors, and Bezier curve descriptors.Fourier descriptors, and Bezier curve descriptors.

To examine the variation of leaf features at different To examine the variation of leaf features at different growth stages.growth stages.

To graphically simulate the growth of seedling To graphically simulate the growth of seedling leaves.leaves.

IMAGE PROCESSING ALGORITHMIMAGE PROCESSING ALGORITHM

NoNo

Leaf image acquisitionLeaf image acquisition

Image binarization and blob analysis

Image binarization and blob analysis

Searching leaf tip and base by discontinuity

Searching leaf tip and base by discontinuity

Boundary edge detectionBoundary edge detection

Determination of basic morphological featuresDetermination of basic morphological features

Bezier curve approximationBezier curve approximation

Petiole designationPetiole designation

Error small enough?

Error small enough?

Determination of Bezier featuresDetermination of Bezier features

Determination of Fourier descriptors

Determination of Fourier descriptors

Bezier curve normalizationBezier curve normalization

Yes

LEAF FEATURE EXTRACTIONLEAF FEATURE EXTRACTION

Conventional morphological featuresConventional morphological features Fourier descriptorsFourier descriptors Bezier featuresBezier features

LEAF FEATURE EXTRACTIONLEAF FEATURE EXTRACTION

Basic quantity descriptorsBasic quantity descriptors• Area (A)• Perimeter (P)• Maximum length (L)• Maximum width (W)• Convex hull (H)

Dimensionless shape factorsDimensionless shape factors• Compactness (C)• Roundness (R)• Elongation (E) • Roughness (G)

Conventional Morphological FeaturesConventional Morphological Features

LEAF FEATURE EXTRACTIONLEAF FEATURE EXTRACTIONConventional Morphological FeaturesConventional Morphological Features

2/4 PAC Compactness

Roundness 2/4 LAR

Elongation LWE /Roughness PHG /

Dimensionless shape factorsBasic quantity descriptors

LL

WW

AA

PP HH

1

0

]/2exp[)(1

)(N

k

NukjksN

ua

)()()( kjykxks

x(k) and y(k) are x-y coordinates of leaf boundary pixels

LEAF FEATURE EXTRACTIONLEAF FEATURE EXTRACTIONFourier descriptorsFourier descriptors

LEAF FEATURE EXTRACTIONLEAF FEATURE EXTRACTIONFourier descriptorsFourier descriptors

Steps to extract Fourier descriptorsSteps to extract Fourier descriptorsFind the major axis of seedling leaf

with Hotelling transform

Find the major axis of seedling leaf with Hotelling transform

Rotate seedling leaf to horizontal positionand select 256 points on the leaf boundary

Rotate seedling leaf to horizontal positionand select 256 points on the leaf boundary

Convert x-y coordinates of boundary pointsto complex number

Convert x-y coordinates of boundary pointsto complex number

Use FFT algorithm to obtain Fourier transform coefficient

Use FFT algorithm to obtain Fourier transform coefficient

Normalization of Fourier transform coefficients to obtain Fourier descriptors

Normalization of Fourier transform coefficients to obtain Fourier descriptors

LEAF FEATURE EXTRACTIONLEAF FEATURE EXTRACTIONFourier descriptorsFourier descriptors

Original Image Binary Image

N=256 N=128 N=64 N=32

N=16 N=8 N=4 N=2

CabbageCabbage

LEAF FEATURE EXTRACTIONLEAF FEATURE EXTRACTIONFourier descriptorsFourier descriptors

Original Image Binary Image

N=256 N=128 N=64 N=32

N=16 N=8 N=4 N=2

LettuceLettuce

LEAF FEATURE EXTRACTIONLEAF FEATURE EXTRACTIONBezier descriptorsBezier descriptors

where m = n – 1, xk+1, yk+1 are the coordinates of the n control points, and Bk,m(u) are the Bezier blending coefficients

m

kkmk

m

kkmk

yuBuy

xuBux

01,

01,

)()(

)()(

kmkkmkmk uu

kmk

muumkCuB

)1(

)!(!

!)1(),()(,

P1

P0

P2

P3Bezier curve

LEAF FEATURE EXTRACTIONLEAF FEATURE EXTRACTIONBezier descriptorsBezier descriptors

Steps to obtain Bezier descriptorsSteps to obtain Bezier descriptors

Image acquisition Image segmentation Boundary detection

Finding leaf tip and leaf base

Fitting boundary withBezier curves

Normalization andobtain bezier descriptors

A B C

D E F

LEAF FEATURE EXTRACTIONLEAF FEATURE EXTRACTIONBezier descriptorsBezier descriptors

Bezier descriptorsBezier descriptors• Leaf tip angleLeaf tip angle• Leaf base angleLeaf base angle• Left control line ratioLeft control line ratio• Right control line ratioRight control line ratio• Normalized control Normalized control

point coordinatespoint coordinates

RESULTSRESULTS

Leaf features at different growth stagesLeaf features at different growth stages• Basic morphologic featuresBasic morphologic features• Bezier descriptorsBezier descriptors

ApplicationsApplications• Geometric Modeling of Seedling LeavesGeometric Modeling of Seedling Leaves• Leaf Shape Comparisons and Plant Leaf Shape Comparisons and Plant

IdentificationIdentification

LEAF FEATURES AT DIFFERENT LEAF FEATURES AT DIFFERENT GROWTH STAGESGROWTH STAGES

Cabbage Seedlings

y = 0.5149x + 8.6391

R2 = 0.954

y = 0.4721x + 7.3878

R2 = 0.981

y = 0.1735x + 2.8094

R2 = 0.935

y = 0.1241x + 2.0504

R2 = 0.964

0

5

10

15

20

25

0 5 10 15 20 25 30

Leaf Area (cm2)

Val

ue

(cm

)

Convex hull perimeterPerimeterLengthWidth

Cabbage Seedlings

y = 0.5149x + 8.6391

R2 = 0.954

y = 0.4721x + 7.3878

R2 = 0.981

y = 0.1735x + 2.8094

R2 = 0.935

y = 0.1241x + 2.0504

R2 = 0.964

0

5

10

15

20

25

0 5 10 15 20 25 30

Leaf Area (cm2)

Val

ue

(cm

)

Convex hull perimeterPerimeterLengthWidth

LEAF FEATURES AT DIFFERENT LEAF FEATURES AT DIFFERENT GROWTH STAGESGROWTH STAGES

Cabbage Seedling

y = 0.0011x + 0.8673

R2 = 0.061

y = 0.0027x + 0.6464

R2 = 0.100

y = 0.0016x + 0.6113

R2 = 0.031

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 5 10 15 20 25 30Leaf Area (cm2)

Val

ue

RoundnessRoughnessCompactness

Cabbage Seedling

y = 0.0011x + 0.8673

R2 = 0.061

y = 0.0027x + 0.6464

R2 = 0.100

y = 0.0016x + 0.6113

R2 = 0.031

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 5 10 15 20 25 30Leaf Area (cm2)

Val

ue

RoundnessRoughnessCompactness

LEAF FEATURES AT DIFFERENT LEAF FEATURES AT DIFFERENT GROWTH STAGESGROWTH STAGES

Cabbage Seedling

y = 0.3077x + 96.2

R2 = 0.016

y = 0.3194x + 140.31

R2 = 0.028

0

50

100

150

200

250

0 5 10 15 20 25 30

Leaf Area (cm2)

Deg

ree

Leaf tip angleLeaf base angle

Cabbage Seedling

y = 0.3077x + 96.2

R2 = 0.016

y = 0.3194x + 140.31

R2 = 0.028

0

50

100

150

200

250

0 5 10 15 20 25 30

Leaf Area (cm2)

Deg

ree

Leaf tip angleLeaf base angle

LEAF FEATURES AT DIFFERENT LEAF FEATURES AT DIFFERENT GROWTH STAGESGROWTH STAGES

Cabbage Seedling

y = 0.0004x + 1.3789

R2 = 0.0006

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0

0 5 10 15 20 25 30Leaf Area (cm2)

Val

ue

Elongation

Cabbage Seedling

y = 0.0004x + 1.3789

R2 = 0.0006

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0

0 5 10 15 20 25 30Leaf Area (cm2)

Val

ue

Elongation

APPLICATIONSAPPLICATIONSGeometric Modeling of Seedling LeavesGeometric Modeling of Seedling Leaves

Wire Frame Model Perspective View Mapping with Texture

Elliptical ModelElliptical Model

APPLICATIONSAPPLICATIONSGeometric Modeling of Seedling LeavesGeometric Modeling of Seedling Leaves

Wire Frame Model Perspective View Mapping with Texture

Bezier Curve ModelBezier Curve Model

Top ViewTop View

Side ViewSide View

Real ImageReal Image Graphics SimulationGraphics Simulation

APPLICATIONSAPPLICATIONS3D Reconstruction of Seedling Structure3D Reconstruction of Seedling Structure

Graphic Simulation of Cabbage SeedlingGraphic Simulation of Cabbage Seedling

APPLICATIONSAPPLICATIONS3D Reconstruction of Seedling Structure3D Reconstruction of Seedling Structure

Top ViewTop View

Side ViewSide View

Real ImageReal Image Graphics SimulationGraphics Simulation

Graphic Simulation of Chinese Mustard SeedlingGraphic Simulation of Chinese Mustard Seedling

APPLICATIONSAPPLICATIONSLeaf Shape Comparisons and Plant Identification Leaf Shape Comparisons and Plant Identification

LeafFeature

Extraction

LeafFeature

Extraction

Leaf Image

MorphologicalFeatures

FourierDescriptors

BezierFeatures

Pattern Recognition

Statistical AnalysisNeural NetworkCluster Analysis

Genetic Algorithm

Pattern Recognition

Statistical AnalysisNeural NetworkCluster Analysis

Genetic Algorithm

PlantIdentification ApplicationsApplications

APPLICATIONSAPPLICATIONSLeaf Shape Comparisons and Plant Identification Leaf Shape Comparisons and Plant Identification

Chinese Mustard

Chinese Heading Cabbage

Cabbage

Lettuce

APPLICATIONSAPPLICATIONSLeaf Shape Comparisons and Plant Identification Leaf Shape Comparisons and Plant Identification

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.0 0.2 0.4 0.6 0.8 1.0roundness

com

pact

ness

Chinese Heading Cabbage

Lettuce

Cabbage

Chinese Mustard0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.0 0.2 0.4 0.6 0.8 1.0roundness

com

pact

ness

Chinese Heading Cabbage

Lettuce

Cabbage

Chinese Mustard

APPLICATIONSAPPLICATIONS

0

20

40

60

80

100

120

140

160

180

0 20 40 60 80 100 120 140 160 180 200

Leaf Tip Angle (degree)

Leaf

Base

Angle

(D

egre

e)

) Chinese Heading Cabbage

LettuceCabbageChinese Mustard

0

20

40

60

80

100

120

140

160

180

0 20 40 60 80 100 120 140 160 180 200

Leaf Tip Angle (degree)

Leaf

Base

Angle

(D

egre

e)

) Chinese Heading Cabbage

LettuceCabbageChinese Mustard

Leaf Shape Comparisons and Plant Identification Leaf Shape Comparisons and Plant Identification

APPLICATIONSAPPLICATIONSLeaf Shape Comparisons and Plant Identification Leaf Shape Comparisons and Plant Identification

0

1

2

3

4

5

6

0 1 2 3 4 5 6

Left Control Line Ratio

Rig

ht

Contr

ol Lin

e R

atio

)

Chinese Head Cabbage

Lettuce

Cabbage

Chinese Mustard0

1

2

3

4

5

6

0 1 2 3 4 5 6

Left Control Line Ratio

Rig

ht

Contr

ol Lin

e R

atio

)

Chinese Head Cabbage

Lettuce

Cabbage

Chinese Mustard

CONCLUSIONSCONCLUSIONS An image processing algorithm was developed An image processing algorithm was developed

to quantitatively describe vegetable seedling leto quantitatively describe vegetable seedling leaf shape. af shape.

The leaf shape descriptors can be classified intThe leaf shape descriptors can be classified into basic morphological descriptors, Bezier curve o basic morphological descriptors, Bezier curve descriptors, and Fourier descriptors.descriptors, and Fourier descriptors.

The Bezier curve can be successfully used to fiThe Bezier curve can be successfully used to fit the leaf boundary of selected vegetable seedlit the leaf boundary of selected vegetable seedlings. Features deduced from Bezier curves, sucngs. Features deduced from Bezier curves, such as leaf tip angle, leaf base angle, normalized h as leaf tip angle, leaf base angle, normalized control points, and control line ratios, can be uscontrol points, and control line ratios, can be used to characterize leaf shape.ed to characterize leaf shape.

The use of Fourier descriptors to model leaf The use of Fourier descriptors to model leaf shape was demonstrated.shape was demonstrated.

The effect of leaf development on the variation The effect of leaf development on the variation of leaf features was investigated. Leaf features of leaf features was investigated. Leaf features invariant to the leaf size were identified.invariant to the leaf size were identified.

The measured features of seedling leaves The measured features of seedling leaves allowed for 3D reconstruction of the vegetable allowed for 3D reconstruction of the vegetable seedling for graphic display and leaf shape seedling for graphic display and leaf shape comparison.comparison.

CONCLUSIONSCONCLUSIONS

THANK YOUTHANK YOU

謝 謝謝 謝