A vision-based parking lot management system

2006 IEEEInternational Conference on Systems, Man, andCybernetics

October8-11, 2006, Taipei, Taiwan

A

Vision-Based Parking Lot Management System

Sheng-Fuu

Lin,Yung-Yao Chen, and

Sung-Chieh

Liu

Abstract-Goals of parking lot management system include counting the number of parked vehicles, monitoring the changes of the parked vehicles over the time, and identifying the stalls

available. To decrease the cost of the production, an integrated

vision-basedsystem isagood choice. In thispaper, we propose a

vision-based parkingmanagement system to manage an outdoor

parkinglotby fourcameras set up atloftofbuildings around it,

sending information, including real-time display, to database of

ITS center via internet. This system enables drivers to find parkingspacesavailableormonitoringtheparking lotwherethey parked their cars easily by wireless communication device. To increase accuracy, in thebeginning, color manage is done to all input images, maintaining color consistency. Then, an adaptive

parking lotbackgroundmodel isgenerated.Theadequate colorof eachparking spaceisfound out using statistical method in color image sequences captured by a camera, and foreground is

extracted based on color information. The result will be further

modified by shadow detection based on luminance analysis. Vision-basedparkingmanagement system can managelargearea

by just several cameras. Adjusting position ofthe camera can

easily make this system suitable for most cases. Besides, this

system is endurable and is easy-installed because of its simple equipment.

I. INTRODUCTION

jarking lotsare animportant object classinmanytraffic and lcivilian applications. In the beginning, a parking lot management systemjust means a charge system which can countthe numberofparkedcarsand theirelapsedtimebypaper cards. However, nowadays, an intelligent parking lot management system, would not simply manage the internal operationofaparkingfacility,butalso should bedesignedtobe compatible with a wide range ofaspects that are intertwined withparking facilities.Undoubtedly, anintelligent

parking

lot management systemplays anessential role inITS

(Intelligent

TransportationSystems).

Forinternal

operation,

a complete parkinglot management systemmustpossessthefunctions of management,surveillance,

This workwassupportedinpartbyNationalScienceCouncil, Taiwan,

R.O.C., under Contract No.EX-91-E-FA06-4-4.

Sheng-FuuLin iswith theDepartmentofElectricalandControlEngineering,

NationalChiao-TungUniversity, Taiwan, R.O.C.

(phone:886-3-5712121Ext.54365;e-mail:[email protected]).

Yung-Yao Chen is with theDepartmentof Electrical and ControlEngineering,

NationalChiao-TungUniversity, Taiwan,R.O.C.

(e-mail:shawn.ece93ggnctu.edu.tw).

Sung-ChichLiu is with theDepartmentof Electricaland ControlEngineering,

NationalChiao-TungUniversity,Taiwan,R.O.C.

(e-mail:liujackO729iyahoo.com.tw).

and control. Goals of parking lot surveillance include counting thenumber of parked motorvehicles, monitoring the changes of the parked vehicles over the time, and identifying the location[1].Advancedvisual-based surveillance systems have toprocess videos coming frommultiple cameras to detect the presence of mobile objects in themonitored scene [2]-[5]. As more methods of identifying simple movements become available, the importance of the contextual methods increases. Activities and movements are not only recognized at the moment of detection, but their interpretation or labeling is affected by the temporally extended context in which the events takeplace. Visualsurveillance requires real-time interpretation or eventclassification of image sequences in order toautomate the detection of predefined alarm situations in a given application domain [6]-[9]. Interesting examples of multisensor surveillance systems have been investigated and discussed in detail[10]-[12]. These systems have been applied successfully in visual security for both indoor and outdoor environments, such as banks, shopping malls, subway stations, and parking lots.

On the other hand, for external operation, a parking lot management system emphasizes the linking and integration of car parks and current traffic so that in-vehicle information systemcan becoordinated. Inmodern prosperous cities, there isagreat need for advanced parking assistantsystems to reduce the hustle for the drivers. Parking Guidance Information System (PGIS), also called as Advanced Public Transport System(APTS), can dynamically collect parking

information,

and provides real-time information service for drivers by the means of on-board guidance system or changeable message board installed on roadside, thus assisting drivers to conveniently andrapidlypark[13]-[14].PGIS wasfirstly used in Aachen, German in 1971, andwas introducedto British in the late of 1970s. Then, PGIS was widely deployedin Japan and other Europe countries such as France. Moreover, the future parking management system is expected to provide advancednavigation service andoptimizedtrafficmanagement system[15]-[16].

Today, many drivers, especially lessexperienced driversor out-of-towners, spend much time finding a

parking

space everyday. Moreover, this situation results in increased travel times andnumberofstops, unexpected delays, greatertravels. Besides,nowadays, asurveillance system isquitecommonon

thewayside or inthe parkinglot. Ifwe have an

algorithm

to

obtainmoreinformationsuchasthenumber and

position

of the unoccupiedparkingspace fromthecamera in a

simple

way,it willhelpdriverstoreduce the time offindingthe

parking

lot

CCD camera may be affected due to different weather condition, thus the whole image looks somewhat bluish or reddish. In orderto prevent color from shifting and makethe systemrobust,the system havetodo automatic whitebalancing toeach incoming image. Whenimplementing colorbalancing algorithm, certain assumptions are sometimes made about the general nature of the color components ofimages. There are a variety of white balancing algorithms. However, "gray world assumption" is used in this paper because of its simplicity. What gray worldassumptionstatesisthat, givenanimagewith sufficientamountof colorvariations, the average value of the red,green, and blue components of the image should average outto a common grayvalue. Thisassumptionis ingeneralvalid since in any given real world scene, it is often the case where lots ofdifferent color variationsaregiven. Since thevariations in color are random and independent, it would be safe to say that given a large enough amount of samples, the average should tend to converge to themeanvalue,which is gray. The adjusted channelgains R', G', and B' can be obtained from

(Ravg +Gavg

+

Bavg)3

Rv a 3

RI

=Rx.

avg

Ravg

G =G x avg Gavg B=Bx avg B_avg

(1)

wherethesubscriptavgrepresents averagevalue.Theresult of usingthe gray worldassumptionis shown inFig. 3(b).

Fig.1. General architecture of theentiresystem.

andtoavoidthejams duetothedriver'slowering speedandthe meaningless search.

Inthispaper, we proposeavision-basedmethodtodeal with theparking lot managementproblem. In Sec. II, to maintain colorstability,grayworldassumptionispreprocessed. Then,a method based oncolor can generate an adaptive background image and extract foreground image. At least, a shadow detection method will modify the foreground image and determine whether the stall is available. Fig. 1 shows the general architecture oftheproposed vision-based parking lot managementsystem.

II. ALGORITHM A. Colorconsistency

Sinceoursystem is processing based on colorinformation, it is necessary that a preprocessing procedure is needed to maintain color stability. However, images acquired from the

(a)

(b)

Fig.2. (a)Theoriginalimage.(b) Result of performing colorconsistency.

B. Backgroundgeneration

Inthissection, an adaptive background will be generated for eachparkingspace. Vehicles are considered as foreground and do foreground extraction to calculate the possibility of being occupiedof eachparkingspace. First, when a camera is set up, we will generate aparking lot model according to the image captured by the camera. In other words, because the position and the angle ofthe camera are fixed, we can define every parkingspace inadvance. Fig. 3 shows the parking lot model.

It seems to be bothersome to locate each space individually. However, a row of parking spaces are usually arranged side by sideandhorizontally. An expression forgenerating aparking lotmodel based on these assumptions may bewrittenas

Bk

(xI y)

=

{(x,y)

f(On,

tp

Wk

Lk)}.

(2)

Thefunction

f

(O,

OP,Wk,Lk)

is given by

x0-

(iksinOfl

+ kcosO6P)

ik=l...

Wk

3

n,Op,WsLk)= h I fp I..-lk= I

f(O0 I~~~L) { o (icos

On+jksinO)

6 lLk

where Bk

(xI

y) are pixels belong to the kth parking space, and

(xo,

yO)

is the starting point ofBk.

0,

and

Op

represent the angle of the parking space, and WJ2V, Lk are the width, length of the kth parking space, respectively. In most cases, a row ofparking spaces are aligned, so

On

and

Op

are usually constants and it is easytogenerateaparking lot by (2).

and the background image is shown in Fig. 4(b). The system canrenew it atregular intervals. Then, in this image, deducing the color of eachunoccupied parking space can be efficiently by using information of the predefined road area Sk

(x,

y)which isnear every parking space. Thus, we can get background image by coloring fitting colors to each parking spaceintheinput image.

(a) (b)

(c)

Fig. 4. (a) Image after (4):the noise of pedestrians would be removed, and the original color of the road is reserved. (b) The background image. (c) Comparing with theoriginal image.

Fig. 3. Parking lot model.

Next,itisimperativetofind thefitting colortoeachpark space. Itcan be difficult to do parking lotmanagement in outdoor parking lot only by a constant background. Th difficulties arise from shadows, changes of sunlight, C

occlusionby other vehiclesorpedestrians. Thegreatvariet~ actual situations necessitates anadaptive background. In f; because it isknownthatinmostparking lots,the color of ri is approximately the same as the color of parking spa determiningthestalls available will be easierif theinformat about the road is knownin advance.Therefore, in the syst animage whichremoves noise ofpedestrians andvehicles the road fromimagesequencescanbegenerated

by

f

(x, y)

=median

{f (x, y)},

n=,I .,

60,

where

fn

(x, y) is intensity (gray scale) value converted fi RGB color space at coordinate (x, y) in the input

image

time n, and f is the outputimage. Since compared with road,pedestriansandvehiclesarerelatively moving

subject

isefficienttousemedianofimagesequencestogetthe

orig

colorof the road. The result ofusing (4)is shown inFig. 4

C.

Foregroundextraction

After generating the background image of every parking space, thevehicles in a color image can bedefined when the RGB values ofbackground pixels significantly differ. So, the foregroundimagecanbeextractedby followingequation

y

of

if

IR

(X,

Y)

-MR

(X

(,

)l

<

UkR

oad

P

1, and

G

(XI Y)

-MG

(XI

Y)I

<

07kG

ice,

,and

ji

(X,

Y)

-MG

(X,

A)

<

J7kB

aion P°) otherwise

em,

on The value of

(7kR

I0kG' kB) will be obtained

by

(4) 07kR =\IA (xR,(XISY -MIR

(XIY)I1

NSk (X,Y)F Sk

rom

e

at

'7kG

=

Z

I

G(XIY)

mkG(XIy)I

9

the NSk (X,y)CSk

5,I

it

2,

tinal

7kB

=

Z

IB(x Y)-mkB(X Y)I

[(a), NSk (X,Y)CSk

(5)

where I is the input image and M is the background image. R, GandB are the color components red, green and blue respectively ofthepixel. Themeaningofmk and Sk aremean value andstandard deviationofeachroadareaSk,which is the predefined road area, and the meaning of

NSk

is total pixel number in Sk The result ofusing (5) to the input image is showninFig. 5.

In fact, most ofthe road is near the color of dark gray. Althoughitisusuallychanging, itis stillinacertain range. For RGBcolormodel,apixel is saidtobegraywiththe property: V-Vg-Vb,so apixelcanbe definedas aforeground pixelif oneof itsR,G, andBpixel values differssignificantly.

thereflectance and irradiance of theobjects. Thus,thestronger thesunlight(orthemorethemeanintensity

f

is),thelower a

and higher ,8 should be chosen. In our system, q = 0.25 and

fo

=145 are

empirical

values. The value of

(rR,

rG,

rB)

will be

obtainedby _R(X,Y) rR

(MR(X,Y)

+

1)

JG(XIY) 8 rG

=M

MG

(

XIY

)

+1)(8 IB(X,Y)

rB

(MB

(X,Y)

+

1)

Themeaning of(7) is to find out whether the deviation of each pixel comes form shadow and remove them, and it is changeablebecause of theparameter

f.

Fig.5. Theforegroundimage.

I

D. Shadow detection and deduction

Inthissection,theerrorfrom shadowwillbedeductedfrom the foreground image using chrominance information. After using foreground extraction, there would besome errorwhich comes from the shadow oftreesbeside the parkingspace. The error comes from failing to distinguish between cars themselves and shadow. Nevertheless, drivers liketopark their carinthe shadow oftreesbecause it is cooler. The shadowwill make thecolor ofparking space changeable, and it will increase the ratio of error. In fact, because it is just caused by the quantityofsunlight,mostofpixel valuesarestillwithin a range. Whenisshadowed, thedegreeofdecreaseofeach RGB color spaceisinproportation.Inotherwords,objects points, like cars, typically haveaninter-framedifference greater than a threshold and it is not unusual that the ratio between shadows and reference image. Therefore,the foreground image extracted by (5)should befurthermodified by

[ ifG( - R)2( )2+ ( )2< I

I)I1(xr

y7

P(x,Y)

= ' d

if

.(X,Y)

[oIesMf

(XIY) Il,otemise

(7)

where f is intensity(gray scale ) value, thesameas(4).And

f isthemean gray level value ofinput image. Parameters a

and ,8 take into account how strongthe light source isw.r.t

(a) (b)

Fig.6. (a) The original image. (b) Image after shadow detection: the pink pixels arethose classifiedasshadow, andwe can seecolor ofcarsremains.

Fig. 7. The modified image.

UsingRGBinformationincreases theaccuracy of the system. Using (7) doesn't define whether it is under shadow clearly. However, itis useful todistinguishthedeviation from shadow and from carsthemselves.Because that kind of deviation is just caused by illumination, most of the input images and their background images are still in a range of the similar chrominance. For each parking space, we can determine whetherthe stalls are available by

R I ,ifp >p

Vark,~~~~~.I*.gla

n. '%-c(et*ttegr0

i$.4--;: I..i...

Oataba.,*%,

Fig. 8. Drivers can get real-time parking lot information by wireless communication device.

(a) (b) (e) (D) 9 4.k4 (c) (d) (g) (h)

(i) (i) (k) (I)

Fig.9. (a)-(d) Experimental results of sunny day,(c)-(h) Experimental results of common weather. (i)-(l) Experimental results of cloudy day. Images fromleftto

rightareinputimage, result ofperformingcolorconsistency, the foregroundimage, and the modified image, respectively. The valueof p will be obtained by

P=N

NB,

(10)

where NB is the total number of pixels which are in the parking spaceBk,

NP,

is the total number ofpixels whose value equalsto 1 inBk, and Rk is the final decision of each

parking space. Parking space Bk is classified as an occupied space if p is sufficient large, in which case, Rk is set to 1. Otherwise,it isclassifiedas aspaceavailable and Rk is set to 0. The choice of the parameter rp is less straight forward and, fornow,isdoneempirically.

-e40

At last, the vision-based parking lot management system calculates Rk and integrates all information. Then, it sends information, which includes counting the number of parked vehicles, monitoring the changes of the parked vehicles over the time, identifying the stalls available to database of ITS center via internetand providingauserinterface where drivers can get real-time parking lot information by wireless communication device. A user interface in this system is shownasFig. 8.

III. EXPERIMENTALRESULTS

The proposed algorithm istestedonimages from cameras. All experimental images are 24-bit color images of size 352 x 288. Fig. 9 shows some images which are under differentweather condition: (1) commonweatherwhose gray level value is from 100 to 130,(2) a cloudy daywhose gray level value is from70 to 120, and(3)asunnydaywhose gray level value is from130 to 170.Theimagesfrom lefttoright,as shown inFigure8,areinputimage, resultofperformingcolor consistency, the foreground image, and the modified image, respectively. Sometimes errors occur, in our experiments, because the horizontal distance between the camera and the vehicle istoolong.Thatis,anangleofelevation from vehicle tothe cameraistoolittle thatittendstobeshelteredbyother vehicles. However, experimental result indicates that this systemisconsiderably preciseand robust ifthecameras are set highenough andneartheparking lot.

IV. CONCLUSION

Avision-basedparkinglotmanagement system isproposed inthis paper. A preprocessing procedure (color consistency) can maintain color consistency and raise stability of this system. Then,generatingan adaptiveparkinglotbackground model based on color and predefined model enables this system suitable in most cases. Shadow detection method increases system accuracy. Thevision-based method makes it possible to manage large area by just several cameras. It is cheap and easy-installed because ofthe simple equipment. However, drivers can get useful real-time parking lot information from this system by wireless communication device. Itisaninteresting topicinthe future worktointegrate this parking management systemwith othersystems, such as trafficsign control system and traffic flow analysis system and itwill improve the traffic greatly.

[3] S. DockstadcrandA.Tekalp, "Multiplecameratracking ofinteracting

and occluded human motion," Proc. IEEE, vol. 89, Issue 10, pp.

1441-1455,Oct.2001.

[4] T. Matsuyama and N. Ukita, "Real-time multitarget tracking by a

cooperativedistributed vision system,"Proc.IEEE,vol. 90, Issue7, pp.

1136-1150,Jul. 2001.

[5] C.Micheloni,G.Foresti,andL.Snidaro,"Anetwork ofco-operative

camerasfor visualsurveillance," Proc. IEE, Vision, Image, andSignal Processing,vol. 152, Issue2,pp.205-212,Apr.2005.

[6] G.L.Foresti,"Areal-timesystemfor video surveillance of unattended outdoorenvironments",IEEE Trans. Circuits and Systemsfor Video Technology, vol. 8, Issue 6,pp.697-704,Oct. 1998.

[7] T.Kanade,R.Collins,A.Lipton,P.Anandan,P. Burt,andL.Wixson,

"Advances in cooperative multi-sensor video surveillance," Proc DARPA ImageUnderstanding Workshop,vol.1,pp.3-24, Nov. 1998.

[8] G.L. Foresti, C. Micheloni, andL.Snidaro, "Event classification for automatic visual-based surveillance of parking lots", Proc. 17th InternationalConferenceonPatternRecognition, vol.3,pp.314-317, Cambridge, UK, Aug.2004.

[9] Yuri Ivanov, Chris Stauffer, Aaron Bobick, andW. E. L.Grimson,

"Video surveillance ofinteractions", Proc. 2nd IEEE Workshopon

VisualSurveillance, pp.82-89,Jun.1999.

[10] I.Haritaoglu,D.Harwood,and L.S.Davis,"W4: real-timesurveillance

ofpeople and their activities," IEEE Trans. Pattern Analysis and MachineIntelligence, vol. 22,Issue8,pp.809-830,Aug.2000. [ 11] U. Urfer, "Integration ofsystemsand services in centralmonitoring

stations(CMS)," Proc. 29th Annu. IEEE International CarnahanConf

SecurityTechnology,pp.343-350, Sanderstead, UK, Oct. 1995.

[12] M.Braasch, M. DiBenedetto, S. Braasch, andR.Thomas, "LAAS

operationsinsupportofairport surfacemovement,guidance, control and surveillance: initialtestresult," Proc. IEEE Position Location and NavigationSymposium,SanDiego, USA,pp.82-89, Mar. 2000. [13] H. Zhong, J.Xu, Y.Tu,Y. Hu, andJ.Sun, "Theresearch ofparking

guidance and information system based on dedicated short range communication," Proc. IEEE Intelligent Transportation Systems, vol.

2,pp.1183-1186,Oct. 2003.

[14] Z. Yang, H.Liu, and X. Wang, "Theresearch on thekeytechnologies for improving efficiency of parking guidance system," Proc. IEEE Intelligent Transportation Systems, vol. 2,pp. 1177-1182, Oct. 2003. [15] M. Wade, K.S. Yoon, and H. Hashimoto, "Development ofadvanced

parking assistance system,"IEEE Trans.IndustrialElectronics,vol.50,

Issue1,pp.4-17, Feb. 2003.

[16] C.C. Li, S.Y. Chou,andS.W.Lin,"Anagent-based platformfordrivers and carparks negotiation," IEEE International Conf. Networking,

Sensing and Control, vol. 2,pp.1038-1043, Taipei, Taiwan,Mar.2004.

REFERENCES

[1] X.WangandA.Hanson,"Parking lot analysis and visualization from aerialimage," Proc.4thIEEE WorkshoponApplications of Computer

Vision,pp.36-41,Oct. 1998.

[2] R.Collins,A.Lipton,H.Fujiyoshi, andT.Kanade, "Algorithms for cooperativemultisensorsurveillance," Proc. IEEE, vol. 89, Issue10,pp. 1456-1477,Oct.2001.