Leaf boundary extraction and geometric modeling of vegetable seedlings

(1)

LEAF BOUNDARY EXTRACTION AND GEOMETRIC LEAF BOUNDARY EXTRACTION AND GEOMETRIC

MODELING OF VEGETABLE SEEDLINGS MODELING OF VEGETABLE SEEDLINGS

Ta-Te Lin, Yud-Tse Chi, Wen-Chi Liao

Department of Bio-Industrial Mechatronics Engi

neering,

National Taiwan University,

Taipei, Taiwan, ROC

(2)

INTRODUCTION



Plant growth measurement and

modeling



Image processing technique



Seedling characteristics

(3)

OBJECTIVES

 To develop image processing algorithms for leaf To develop image processing algorithms for leaf

boundary extraction.

 To model leaf boundary with Bezier curves and To model leaf boundary with Bezier curves and

develop leaf features based on Bezier curve.

 To determined leaf features of selected vegetable To determined leaf features of selected vegetable

seedlings based on basic morphological descriptors,

Fourier descriptors, and Bezier curve descriptors.

 To examine the variation of leaf features at different To examine the variation of leaf features at different

growth stages.

 To graphically simulate the growth of seedling To graphically simulate the growth of seedling

leaves.

(4)

IMAGE PROCESSING ALGORITHM

No

Leaf image acquisition

Image binarization and blob analy sis

Searching leaf tip and base by discontinuity

Boundary edge detection

Determination of basic morphological features

Bezier curve approximation

Petiole designation Petiole designation Error small enough? Error small enough?

Determination of Bezier features

Determination of Fourier descriptors

Bezier curve normalization

Bezier curve normalization Yes

(5)

LEAF FEATURE EXTRACTION



Conventional morphological features



Fourier descriptors

(6)

LEAF FEATURE EXTRACTION



Basic quantity descriptors

• Area (A)

• Perimeter (P)

• Maximum length (L)

• Maximum width (W)

• Convex hull (H)



Dimensionless shape factors

• Compactness (C)

• Roundness (R)

• Elongation (E)

• Roughness (G)

Conventional Morphological Features

(7)

LEAF FEATURE EXTRACTION

Conventional Morphological Features

2

/

4 A

P

C





Compactness Roundness

_R

_

₄

_A

_/



_L

2 Elongation

E



W

/

L

Roughness

G



H

/

P

Dimensionless shape factors Basic quantity descriptors

L L W W A A P P HH

(8)



 





1 0

]

/

2 exp[

)

(

1 )

(

N k

N

uk

j

k

s

N

u

a



)

(

)

(

)

(

k

x

k

jy

k

s





x(k) and y(k) are x-y coordinates of leaf boundary pixels

LEAF FEATURE EXTRACTION

Fourier descriptors

(9)

LEAF FEATURE EXTRACTION

Fourier descriptors



Steps to extract Fourier descriptors

Find the major axis of seedling leaf with Hotelling transform Find the major axis of seedling leaf

with Hotelling transform

Rotate seedling leaf to horizontal position and select 256 points on the leaf boundary Rotate seedling leaf to horizontal position and select 256 points on the leaf boundary

Convert x-y coordinates of boundary points to complex number

Use FFT algorithm to obtain Fourier transform coefficient Use FFT algorithm to obtain

Fourier transform coefficient

Normalization of Fourier transform coefficients to obtain Fourier descriptors

(10)

LEAF FEATURE EXTRACTION

Fourier descriptors

Original Image Binary Image

N=256 N=128 N=64 N=32

N=16 N=8 N=4 N=2

Cabbage

(11)

LEAF FEATURE EXTRACTION

Fourier descriptors

Original Image Binary Image

N=256 N=128 N=64 N=32

N=16 N=8 N=4 N=2

Lettuce

(12)

LEAF FEATURE EXTRACTION

Bezier descriptors

where m = n – 1, x_k+1, y_k+1 are the coordinate

s of the n control points, and B_k,m(u) are the

Bezier blending coefficients



   



m k k m k m k k m k

y

u

B

u

y

x

u

B

u

x

0 1 , 0 1 ,

)

(

)

(

)

(

)

(

k m k k m k m k

u

k

m

k

m

u

m

k

C

u

B









(

1 )

)!

(!

!

)

1 (

)

,

(

)

(

, P₁ P₀ P₂ P₃ Bezier curve

(13)

LEAF FEATURE EXTRACTION

Bezier descriptors

Steps to obtain Bezier descriptors

Image acquisition Image segmentation Boundary detection

Finding leaf tip and leaf base

Fitting boundary with Bezier curves

Normalization and obtain bezier descriptors

A B C

(14)

LEAF FEATURE EXTRACTION

Bezier descriptors



Bezier descriptors

• _{Leaf tip angle}

_{Leaf tip angle}

• Leaf base angle

_{Leaf base angle}

• Left control line ratio

_{Left control line ratio}

• Right control line ratio

_{Right control line ratio}

• _{Normalized control}

_{Normalized control}

point coordinates

(15)

RESULTS



Leaf features at different growth stages

• Basic morphologic features

_{Basic morphologic features}

• Bezier descriptors

_{Bezier descriptors}



Applications

• _{Geometric Modeling of Seedling Leaves}

_{Geometric Modeling of Seedling Leaves}

• Leaf Shape Comparisons and Plant

_{Leaf Shape Comparisons and Plant}

Identification

(16)

LEAF FEATURES AT DIFFERENT

GROWTH STAGES

Cabbage Seedlings y = 0.5149x + 8.6391 R2 = 0.954 y = 0.4721x + 7.3878 R2 = 0.981 y = 0.1735x + 2.8094 R2 = 0.935 y = 0.1241x + 2.0504 R2 = 0.964 0 5 10 15 20 25 0 5 10 15 20 25 30 Leaf Area (cm2) V al u e (c m )

Convex hull perimeter Perimeter Length Width Cabbage Seedlings y = 0.5149x + 8.6391 R2 = 0.954 y = 0.4721x + 7.3878 R2 = 0.981 y = 0.1735x + 2.8094 R2 = 0.935 y = 0.1241x + 2.0504 R2 = 0.964 0 5 10 15 20 25 0 5 10 15 20 25 30 Leaf Area (cm2) V al u e (c m )

Convex hull perimeter Perimeter

Length Width

(17)

LEAF FEATURES AT DIFFERENT

GROWTH STAGES

Cabbage Seedling y = 0.0011x + 0.8673 R2 = 0.061 y = 0.0027x + 0.6464 R2 = 0.100 y = 0.0016x + 0.6113 R2 = 0.031 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0 5 10 15 20 25 30 Leaf Area (cm2) V al u e Roundness Roughness Compactness Cabbage Seedling y = 0.0011x + 0.8673 R2 = 0.061 y = 0.0027x + 0.6464 R2 = 0.100 y = 0.0016x + 0.6113 R2 = 0.031 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0 5 10 15 20 25 30 Leaf Area (cm2) V al u e Roundness Roughness Compactness

(18)

LEAF FEATURES AT DIFFERENT

GROWTH STAGES

Cabbage Seedling y = 0.3077x + 96.2 R2 = 0.016 y = 0.3194x + 140.31 R2 = 0.028 0 50 100 150 200 250 0 5 10 15 20 25 30 Leaf Area (cm2) D eg re e

Leaf tip angle Leaf base angle

Cabbage Seedling y = 0.3077x + 96.2 R2 = 0.016 y = 0.3194x + 140.31 R2 = 0.028 0 50 100 150 200 250 0 5 10 15 20 25 30 Leaf Area (cm2) D eg re e

Leaf tip angle Leaf base angle

(19)

LEAF FEATURES AT DIFFERENT

GROWTH STAGES

Cabbage Seedling y = 0.0004x + 1.3789 R2 = 0.0006 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 0 5 10 15 20 25 30 Leaf Area (cm2) V al u e Elongation Cabbage Seedling y = 0.0004x + 1.3789 R2 = 0.0006 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 0 5 10 15 20 25 30 Leaf Area (cm2) V al u e Elongation

(20)

APPLICATIONS

Geometric Modeling of Seedling Leaves

Wire Frame Model Perspective View Mapping with Texture

Elliptical Model

(21)

APPLICATIONS

Geometric Modeling of Seedling Leaves

Wire Frame Model Perspective View Mapping with Texture

Bezier Curve Model

(22)

Top View

Side View

Real Image

Real Image Graphics SimulationGraphics Simulation

APPLICATIONS

3D Reconstruction of Seedling Structure

Graphic Simulation of Cabbage Seedling

(23)

APPLICATIONS

3D Reconstruction of Seedling Structure

Top View

Side View

Real Image

Real Image Graphics SimulationGraphics Simulation

Graphic Simulation of Chinese Mustard Seedling

(24)

APPLICATIONS

Leaf Shape Comparisons and Plant Identification

Leaf Feature Extraction Leaf Feature Extraction Leaf Image Morphological Features Fourier Descriptors Bezier Features Pattern Recognition Statistical Analysis Neural Network Cluster Analysis Genetic Algorithm Pattern Recognition Statistical Analysis Neural Network Cluster Analysis Genetic Algorithm Plant

(25)

APPLICATIONS

Chinese Mustard Chinese Heading Cabbage

Cabbage

(26)

APPLICATIONS

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.0 0.2 0.4 0.6 0.8 1.0 roundness co m pa ct ne ss

Chinese Heading Cabbage Lettuce Cabbage Chinese Mustard 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.0 0.2 0.4 0.6 0.8 1.0 roundness co m pa ct ne ss

Chinese Heading Cabbage Lettuce

Cabbage

(27)

APPLICATIONS

0 20 40 60 80 100 120 140 160 180 0 20 40 60 80 100 120 140 160 180 200

Leaf Tip Angle (degree)

Le af B as e A ng le ( D eg re e)

) _{Chinese Heading Cabbage}

Lettuce Cabbage Chinese Mustard 0 20 40 60 80 100 120 140 160 180 0 20 40 60 80 100 120 140 160 180 200

Leaf Tip Angle (degree)

Le af B as e A ng le ( D eg re e)

) _{Chinese Heading Cabbage}

Lettuce Cabbage

Chinese Mustard

(28)

APPLICATIONS

0 1 2 3 4 5 6 0 1 2 3 4 5 6

Left Control Line Ratio

R ig ht C on tr ol L in e R at io )

Chinese Head Cabbage Lettuce Cabbage Chinese Mustard 0 1 2 3 4 5 6 0 1 2 3 4 5 6

Left Control Line Ratio

R ig ht C on tr ol L in e R at io )

Chinese Head Cabbage Lettuce

Cabbage

(29)

CONCLUSIONS

 An image processing algorithm was developed An image processing algorithm was developed

to quantitatively describe vegetable seedling le

af shape.

 The leaf shape descriptors can be classified intThe leaf shape descriptors can be classified int

o basic morphological descriptors, Bezier curve

descriptors, and Fourier descriptors.

 The Bezier curve can be successfully used to fiThe Bezier curve can be successfully used to fi

t the leaf boundary of selected vegetable seedli

ngs. Features deduced from Bezier curves, suc

h as leaf tip angle, leaf base angle, normalized

control points, and control line ratios, can be us

ed to characterize leaf shape.

(30)

 The use of Fourier descriptors to model leaf The use of Fourier descriptors to model leaf

shape was demonstrated.

 The effect of leaf development on the variation The effect of leaf development on the variation

of leaf features was investigated. Leaf features

invariant to the leaf size were identified.

 The measured features of seedling leaves The measured features of seedling leaves

allowed for 3D reconstruction of the vegetable

seedling for graphic display and leaf shape

comparison.

CONCLUSIONS

(31)