Prediction of lane tendency - G EOMETRIC L ANE M ODEL

CHAPTER 4 LANE DETECTION

4.2 G EOMETRIC L ANE M ODEL

4.2.2 Prediction of lane tendency

(4-9)

east-squares approximation. Accordingly, the

image domain can be obtained from (4-2).

On the other hand, since u is a function of v in (4-2), it is denoted as . The

first in powers of

(4-15) If some data of the pair (u_L, ) cients, C_xy₀, C_xy₁, and C_xy₂, of (4-10) can be determined by means of the weighted-l

unknowns (k ,m ,b) of (4-2) can be further confirmed so that the lane tendency in the be inaccurate, so that there is a certain error in the lane tendency in the image domain predicted from (4-2). N les , few data still contribute the info

riva

are given, the param

everthe s rmation to the lane

cy in a small specific region of the image is just expected, (4-16) can be performed to obtain the approxim tion by expanding about the

coor cific region of the image.

t and right sides of the lane ca cing with

(u_L u_R

tendency around them. If the lane tenden

a f

dinate v close to the spe_e

Eventually, note that both lef n also be predicted from (4-16) b b mW 2, respectively, i.e.

4.3 M

The task of marking detection is to detect such marking pixels lying on both sides of the

lane in the r can be characterized by two in

(1)

e marki as t rod hig

(2) system

e. An e rated in Fig. 4.3

s could slightly vary in diffe

e considered the same. In general, the marking width ranges from 10 cm to 30 cm, and is referred to 20 cm in this thesis. Since the lane markings will be detected in the image coordinates, the constant marking width in the world domain in the image domain from (2-32), as

arking Detection

oad image. The lane markings trinsic factors:

The gray levels of the markings are greater than those of the road surface. There exist the sharper edges between th ngs and the road surface, so o p uce the her gradients located at the edges. Since only the vertical edges are interesting, the 3×3 mask shown in Fig. 4.2 is used to compute the gradients. Notice that the maximum gradients are located at the darker pixels.

All widths of the markings are thought constant in the world coordinate since they are artificially painted on the road surfac xample is demonst

Of course the marking width rent areas, but all of them in a certain zone can b

can be transformed into its corresponding width described in Section 2.2.2. See Fig. 4.3.

1 2

1 −

1 2

1 −

1 2

1 −

Fig. 4.2 The 3×3 mask for determining the gradients of the vertical edges. Note that the maximum gradients will be located at the darker pixels.

(a) The original road image.

(b) The marking in system.

Fig. 4.3 The constant marking width in the world coordinate system. (a) The original road image. (b) The marking in the image coordinate system. (c) The top view of (a). (d) The marking in the world coordinate system. Its width M_W is approximately constant.

the image coordinate

the world coordinate

constant.

) The marking in system. Its width is

detect the marking, as illustrated in Fig. 4.4.

(a) Fig. 4.4 (a) shows the marking with the greater intensity and its horizontal profile for a given scanning line. It is clear that the intensity of the marking is greater than that of the road surface.

(b) Due to the higher brightness on the marking, the detection is base

of horizontal dark-light-dark (DLD) intensity transitions [16]. In t the point M is said to be situated at the location of the DLD-transition if its intensity is greater than those at its horizontal left

For a given scanning line, this process sear the DLD-transition u ound out or the scanning arrives at its ending. Go to step (c) if one transition is determined, or else exit.

(c) Two maximum gradients and within the intervals of and are determined at the points L and R respectively by the mask in Fig.

a possible marking region

d on the determination his thesis

m , see Fig. 4.4 (b).

and right neighbors by a distance ches for the location of ntil one transition is f

GL G_R

[

M −m_i/2, M

)

go back to step (b). The threshold LRth is related to the mini ble marking width in the image domain, and usually it is the half of . the marking is determined and thus return the center of

ectiv

(

L,R

)

. If not, there could be some deeper valleys within

(

L,R

)

as figured in Fig. 4.4 (e). In the unsuccessful case, the flow goes back to step (c) in order to determine a new possible marking region. A ew maximum gradient is detected within

(

L,M

)

to replace if the mean intensity

ithin is less than that within

(

L,M

) (

M ,R

)

, or else it is detected within to place , which yields a new possible marking region. Proceeding in a similar shion, the process will exactly evaluate the marking for a given scanning line.

(

M ,R

)

re fa

(a)

Fig. 4.4 Steps of the marking detection.

Searching for the point M satisfying I

Two maximum gradients G

L and

G within the intervals of

[

M −m_i/2, M

)

and

tively, and th ).

(

M, M +m_i/2

]

are located respec us

enclose a region (L, R

Yes

The marking with the greater intensity than that on the road surface.

The horizontal profile of the ideal marking for a given scanning line.

4.4 Lane Detection in the Single Mode

4.4.1 Overview

Given a single frame, the topic in this procedure is to determine the lane without the aid of the last frame. In the beginning, there is no information about the lane. However, according to (4-1), the lane has been modeled as a quadratic polynomial with the coefficients (k ,m ,b), whose probable ranges are as follows [14, 18]:

)

Fig. 4.5 displays the probable marking ranges of both left Therefore, the area of marking detection at the initia

and right sides of the lane.

l phase is restricted to these two ROIs.

Fig. 4.5 The possible ranges of the markings on both sides of the lane at the initial state.

Since the probable ranges in the bottom of the image are the narrowest, the image plane is divided into n zones ordered according to the direction of the v-axis and the size of the first zone is greater, as shown in Fig. 4.6. The marking detection is performed zone by zone, from bottom to top in the image.

ocess is iterated for each zone from bottom to arkings on both sides of the lane are found out, and the parameters can be

det the detection results and

The details will be described in the following.

Fig. 4.6 The image is divided into n zones, and the markings are detected from bottom to up.

As soon as the left and right markings in a certain zone are determined, an estimate of )

, ,

(k m b is evaluated from (4-10), and the probable left and right markings in the next zone can be predicted by (4-16) and (4-18); moreover, the ROIs can be set up in the detection zone to narrow the searching area. After the same pr

up, the m (k ,m ,b)

ermined. Fig. 4.7 demonstrates the predicted ranges for each zone.

○⁰

○⁵

○⁴

○³

○²

○¹

(b)

(c)

Fig. 4.7 to up,

respectively

spectively.

(a)~(f) are the intermediate phases where the zones are detected from bottom . The black solid curve comes from (4-2) while the white dashed line is the approximation of (4-16). The left and right dotted blocks are the detection regions of interest in the next zone. (a), (b), and (c) show the detection ROIs in the zones 1, 2, and 3, re

(d)

(e)

(f)

Fig. 4.7 (a)~(f) are the intermediate phases where the zones are detected from bottom to up, respectively. The black solid curve comes from (4-2) while the white dashed line is the approximation of (4-16). The left and right dotted blocks are the detection regions of interest in the next zone. (d) and (e) show the detection ROIs in the zones 4 and 5, respectively. Fig. (f) shows the final detection result, and it is obvious that the rears of the detected and predicted lines match with each other.

4.4.2 Detection flow

Fig. 4.8 shows the flowchart of lane detection in the single mode. In the beginning, the initial possible ranges are searched zone by zone, and the zone is scanned row by row, from bottom to up, to detect the markings. If both marking points on the left and right sides are found out and the distance between them is valid, this procedure is terminated after finishing the current zone. Since the constant lane width in the world domain is assumed, its

esponding width in the image domain can be obtained from (2-32). Thus the distance between the left and right detected pointes is said valid if it approximates to the constant lane width in the image coordinate system.

After the lane detection at the initial state, the next zone is located. The probable regions of the markings in this zone can be predicted by applying (4-16) and (4-18), where comes from the coordinate v near the current zone. The ROIs are determined at the process, namely Specify ROI, to narrow the searching ranges. Both markings in these two ROIs are searched and determined. And then the detection results for both markings are processed at t e

ely Decision Tree. Both the processes of Specify ROI and Decision Tree will be interpreted in next sections. Proceeding in the same way, the lane can be confirmed after all zones are detected.

corr

h procedure, nam

Fig. 4.8 The flowchart of lane detection in the single mode.

START

End of ? v Lane Detection at the initial phase

Specify ROI

End of zone ? Marking Detection

Decision Tree zone zone+1←

Lane Prediction

END v ← v + 1

4.4.3 Specify the detection region of interest

This proced Is) for the marking

MAIN and SUB, initialized to e called the Decision Tree.

(1) pROI = SUB

In this case, the left and right ROIs of the current ordinate are defined as:

ure is to specify the left and right regions of interest (RO

detection, and these ROIs depend on the parameter pROI, composed of properties MAIN, and determined by the procedur

[

u_i _s m_i u_i _s m_i

]

ROI = ₋₁−λ ⋅ , ₋₁+λ ⋅ (4-20)

where is the abscissa detected in the last ordinate is the constant marking width in the image domain as described in Section 4.3, and

−1

ui v_i₋₁, m_i

λs is a constant. Since the abscissa is the target to detect, its probable region is specified in the neighborhood of

cissa d the ma

(2) pROI = MAIN

In this condition, the left and right ROIs of the current ordinate are defined as:

the last detected abs u_i₋₁ ue to the continuity of rking.

[

u_i _m m_i u_i _m m_i

]

ROI = −λ ⋅ , +λ ⋅ (4-21)

where the abscissa corresponding to the current ordinate is evaluated in the process of the lane prediction, and

ui v_i

λm is a constant. Since the last abscissa is not detected, a guess about can com from the combination of (4-16) and (4-18), and a greater

−1

ui e

λm than λ_s is used due to the unknown abscissa of the lane sides.

4.4.4 Decision tree

fter the marking detection is performed in both left and right specified regions of

interest for a giv a s stated below

will happen. This process is to judge whether both left and right detected marking points belong to the boundaries of the lane, and to update the control parameters if true. Two control

−1

en ordin te v_i, as illustrated in Fig. 4.8, one of four cases a

parameters, namely pROI and pMode respectively, will be updated in this process. The control parameter pROI is the enumeration composed of two identifiers, namely MAIN and SUB, and it is u

marking llustrates the flowchart of the Decision Tree, and four conditions are discu

(1) Bo d:

s approximation.

arking is found:

ode =

LE rocess called the Update Left

th eig

etected, and that the current marking point in the neighborhood of the last marking point is found out; we believe two points belong to the same marking thanks to the marking continuity.

(3)

sed to decide the ROIs for the marking detection, as proposed in Section 4.4.3. The other control parameter pMode is the enumeration composed of four identifiers, namely BOTH, LEFT, RIGHT, and NONE, and it responses the result of detecting both left and right

points. Fig. 4.9 i ssed as follows:

th markings are foun

In this case, both left and right marking points are found out, and the distance between them is compared to the constant lane width in the image dom in. If it is legal, the lane width in the world domain is updated and both marking points are added to the fitting data accompanied with a greater weight for the weighted-least-square

(2) Only the left m

In this situation, only the left marking point is found out. If pROI = SUB or pM FT, proceed to the p . In the process of Update Left, the right marking point can be estimated as what is the left point plus the lane width;

afterward both points are added to e fitting data with a smaller fitting w ht, and finally pROI is assigned as SUB. The conditions of pROI = SUB or pMode = LEFT are based on the continuity of the marking, and they mean that the last marking point is d

Only the right marking is found:

In this condition, if pRO = SUB or pMode = RIGHT, then the left marking point is approximated as what is the right point minus the lane width, and the process of Update Right similar to the case (2) is driven.

Since no marking is found in this case, no point is added to the fitting data. We just assign pROI and pMode as MAIN and NONE, respectively.

Fig. 4.9 The flowchart of Decision Tree.

Marking Detection

RIGHT pMode

SUB

pROI ∨

Update Right

pROI = MAIN pMode = RIGHT LEFT

pMode SUB pROI

∨

Update Left

pROI = MAIN pMode = LEFT Valid Width

Update Both

pROI = MAIN pMode = BOTH Both ?

Left ?

Right ?

pROI = MAIN pMode = NONE

(None)

END

4.5 Lane Detection in the Successive Mode

ith a prior knowledge of the ne detected on the last frame. Due to the slight variation between two successive frames, the can be regarded as an estimate about the lane on this frame. In addition, both estimated boundaries of the lane on the current frame can also be evaluated from the estimated lane.

Fig. 4.10 The flowchart of lane detection in the successive mode.

Fig. 4.10 shows the flowchart of the lane detection in the successive mode. The main regions of interesting markings can be specified by the neighborhood of the estimated lane boundaries. And thus the lane on the current frame can be determined by detecting directly the main or sub ROIs in the same way as mentioned in the last section. After finishing scanning

The purpose in this successive mode is to detect the lane w la

) , , (k m b last determined lane parameters

START

End of v ? Specify ROI

Marking Detection

Decision Tree

END v ← v + 1

arameters can, for the purpose of the robust detection, be evaluated from the data of a mixture of the current and last mark

succes

ing time, the algorithm in the successive mode should be simple and effective as w

4.6 Update of

The update of lane param topics. At first, the lane tendency is

necessary in order to warn the dr art vehicle with the

lane information fo nks to the detection

algorithm as a quadratic curve with

parameters , is determined as soon as the detection finishes. However, the physical lane parameters, such as the offset, the orientation, or the curvature, are usually desired in the practical applications, and they will be dis ction 4.6.2.

On the other hand, the second topic is concerning the 3-D reconstruction of the lane. In words, the goal here is to reconstruct the lane information about the road inclination and the lane width for the next detection stage or the v troller.

The algo dition of the

constant lane width. Nevertheless, due to the vibration in motion, the variation of the road inclination, or the illega

3-D lane parameters. This will be presented in Section 4.6.1.

) , , (k m b between two successive frames is assumed, the current lane p

ing points, where the marking points on the last frame are assigned a smaller fitting weight.

The process is repeated in the sive mode until it fails, and then returns to the single mode. The detection in the successive mode is dominant since it takes the majority, and therefore, for reducing the process

e do.

Lane Parameters

eters involves two

ivers in bad situations or to supply the sm r the purpose of tracking the lane automatically. Tha involving the prediction phase, the lane, modeled

) , , (k m b

cussed in Se

ehicle con

rithm of lane detection proposed in this thesis is based on the con

l assumption of the constant lane width, some errors may exist in the results of the formulas deduced from the computer vision. Thus, it is necessary to calibrate the

4.6.1 3-D reconstruction

Applying (4-7) and (4-8) to (2-27) and rearranging it by (4-15), it yields

[ ]

Using the weighted-least-squares method, the coefficients of (4-22), i.e.

e, which are listed

4.6.2 Offset, orientation, and curvature

Since the lane has been modeled as a quadratic polynomial with the coefficients )

, ,

(k m b , from the fundamental calculus it is easy to obtain the offset, orientation, and curvature of the lan as follows:

Chapter 5 Experimental Results

5.1 Results of Obstacle and Lane Detection

In the obstacle and lane detection system, two cameras are mounted top and bottom on

ou d both road images are captu

e results of obstacle and lane detection when the experimental vehicle is running on the expressway and freeway with the velocities of 80 km/hr and 110 km/hr, respectively. It is

clear th d can be deter

ked

r experimental vehicle, an red simultaneously. Fig. 5.1 shows the results of obstacle detection on a hill road. On the other hand, Fig. 5.2 and 5.3 display th

at the vehicles on the roa mined. In addition, the roadsides such as the median can also be mar .

Fig. 5.1 Results of obstacle detection on a hill road.

Fig. 5.2 Results of obstacle and lane detection on the expressway.

Fig. 5.3 Results of obstacle and lane detection on the freeway.

detection system.

Several road conditions, such as the straight or crooked cases, shadows or sunlight conditions, he results are very satisfactory, as displayed in Fig. 5.4~5.8. The shadows on the road surface will result in the variation of the brigh

Fig. 5.4 Results of lane detection on the straight roads.

performed alone using a single monochromatic camera without the obstacle

the roads interfered with the text or vehicles, are tested, and t

tness. The gray values of the texts on the road surface are similar to those of the lane markings. In addition, the traffic in downtown is usually heavy so that the markings are often covered by vehicles. The proposed algorithm can work in all cases, even if only left or right lane side is available. On the other hand, the lane detection system can also be performed in the night or rainy environment, as shown in Fig. 5.9 and 5.10 respectively. The results demonstrate the proposed algorithm is very robust.

Fig. 5.6 Results of lane detection on the roads with shadows or the sunlight.

Fig. 5.5 Results of lane detection on the crooked roads.

Fig. 5.7 Results of lane detection on the roads interfered with the text.

Fig. 5.8 Results of lane detection on the roads affected by the vehicles.

(a) The original road image.

(b) The detection result of (a).

Fig. 5.9 Result of lane detection on the night road. (a) the original night road image. (b) th detection result of (a).

(b) The detection result of (a).

Fig. 5.10 Result of lane detection on the rainy road. (a) the original rainy road image. (b) the detection result of (a).

F experimental results of real-time lane detection on our experimental vehicle, namely TAIWAN iTS-1 as shown in Fig. 13, running on the freeway under 110 km/hr. The road image sequence of size

ig. 5.11 and 5.12 display the

493

644× is captured with the frame rate of 30 fps by the Domino Alpha 2 board and the Hitachi KP-F3 CCD camera mounted on the smart vehicle, and then the image is processed by the proposed algorithm of lane detection

在文檔中基於電腦視覺之即時穩健的泛型障礙物與車道偵測行車系統 (頁 69-0)