智慧型停車場系統:車牌辨識,車輛導引,及裝置控制子系統

全文

(1)Chapter 1 Introduction. 1.1 Motivation As humans adapt to an ever-changing environment, especially one where personal vehicles have become a necessity rather than a luxury, so will we need to be able to better manage this growing means of transportation. According to statistics from the Ministry of Transportation. and. Communications, as. of. 2008. there. were. more. than. 21,092,385 registered vehicles of all different makes in Taiwan. Comparing this to the population of Taiwan (23,037,031 people), we can see that on average, almost every person owns a vehicle; this confirms the notion that when it comes to transportation, people prefer using their own vehicles for convenience. Cars make life more convenient and bring people closer.. However, cars also bring many related problems, one of which. is the difficulty of finding a parking space for vehicles, especially in very populated cities such as Taiwan. There are four common means of parking that we may experience in our day to day life: roadside parking, off-street parking, parking towers, and multi-storey car parks. Of these different kinds of parking, roadside parking occupies the roadside and hampers the flow of traffic. As a result, it is difficult to develop in cities with heavy traffic.. Similarly,. off-street parking needs large amounts of land, something that is sparsely available in metropolitan cities. It is, however, known to work well in the more suburban areas, such 1-1.

(2) as the parking lots outside of Taipei Zoo or along the Tamshui riverside. From statistics gathered from the Taipei City Department of Transportation and the Transportation Bureau of Kaohsiung City (Tables 1.1 and 1.2), we can see that there is hardly any growth in roadside parking and off-street parking.. Table 1.1 Numbers of parking spaces in Taipei Parking Year. Roadside Off Street Private. Affiliated to Independent Total Buildings. 90. 44,544. 34,991. 2,276. 295,522. 11,412. 388,745. 91. 43,672. 38,414. 2,768. 317,660. 13,981. 416,495. 92. 43,256. 40,019. 1,464. 339,223. 16,793. 440,755. 93. 44,947. 39,935. 2,327. 360,504. 18,689. 466,402. 94. 47,640. 38,193. 4,938. 381,078. 20,365. 492,214. 95. 48,417. 38,675. 5,527. 401,536. 23,379. 517,534. 96. 49,378. 39,165. 5,383. 422,765. 23,857. 540,548. 97. 48,942. 39,478. 5,977. 448,293. 26,931. 569,621. Reference: Taipei City Department of Transportation Website. Table 1.2 Numbers of parking spaces in Kaohsiung Parking Year. Public Private Affiliated to Roadside Scenic Spot off Street off Street Buildings. Total. 90. 6,786. 15,112. 43,092. 149,917. 1,110. 216,017. 91. 8,887. 20,800. 43,223. 155,595. 1,110. 229,615. 92. 9,153. 21,504. 41,183. 157,225. 1,110. 230,175. 93. 11,202. 22,226. 13,895. 164,431. 2,457. 214,211. 94. 12,015. 23,543. 45,112. 180,211. 2,235. 263,116. 95. 13,398. 23,294. 45,645. 196,488. 2,462. 281,287. 96. 13,890. 25,862. 45,639. 205,053. 2,462. 292,906. Reference: Transportation Bureau, Kaohsiung City Website. It follows then, that the main methods of parking in metropolitan areas will be parking towers, multi-storey car parks or underground parking lots. These parking spaces are affiliated to buildings and, according to Tables 1.1 and 1.2, make up 1-2.

(3) more than 70% of the total parking spaces available. The competitive advantage of parking towers is that they can cater for more vehicles, while occupying a smaller land area. In other words, parking towers such as Taipei Bade Car Park and Guangfu Car Park 2, can house more vehicles than multi-storey car parks of the same size. However, the mechanical operation of parking towers has to be controlled by attendants where vehicles enter and exit these parking towers. This is usually a very time-consuming process, and is one of the main reasons why parking towers are usually small and have only a limited number of parking spaces. On the other hand, multi-story car parks like Luoyang Car Park, large underground parking lots such as Fuqian Car Park, and the underground parking lots usually present at schools such as NTNU, generally have larger areas and can contain more vehicles. These car parks can presently be divided into three types of management systems:  Normal Parking Management Systems, such as those designed by Beijing Saikang Co. Ltd (http://www.saikang.com.cn/).  Automatic Parking Management Systems, such as those developed by Sunhope Co. (http://www.sunhopepark.com.tw), and  Intelligent Parking Management Systems, such as those designed by TagMaster (http://www.tagmaster.com). The principles of how these parking management systems work are described in the following sections. Normal Parking Management Systems, like those designed by Beijing Saikang Co. Ltd, use parking ticket machines to charge parking fees. When entering the car park, the ve hi c l e ’ sowne ri sr e qui r e dt or e t r i eve a ticket with their entrance time. When the vehicle leaves the car park, the total time spent in the car park is calculated, and a payment is requested. Using this type of parking system, only the timestamp of a vehicle entering the 1-3.

(4) car park is recorded, while other identificatory information on the vehicle is not. The onus is also on the owner to keep their parking tickets, or complications may arise in calculating the required parking fee. Automatic management systems, such as that used at the NTNU underground car park, use ID cards for management. The inductive ID card, which is given to each registered vehicle owner, is scanned by sensors at both the entrance and exit of the car park. These ID cards contain information that is used to identify both the owner and the vehicle as well as keeping a time stamp of the parking times. Using these systems, owners are not required to stop their vehicle at the entrance gate every time to pick up a ticket. Instead, they are allowed inside by the entrance gate automatically upon detection of the ID card. The downside of this system is that owners must apply for an ID card before they enter the car park, while for temporary parking, car owners are still required to pick up their tickets, keep them and pay a fee according to the tickets. Intelligent parking management systems, such as those designed by TagMaster, are controlled and managed by IC-Chips, and are widely considered to be the best of the three systems. The customer information is initially recorded onto their IC-Chip upon application to use the car park. Then, when the vehicle enters the parking area, the sensor reads and transfers the information from the IC-Chip to the management system, for vehicle identification.. As for space management, these systems have more detailed. vacancy management concepts. The parking conditions in each section can be broadcast to the owners of IC-chips, informing customers of their nearest and most convenient car space as soon as they enter the car park. The downside of this system is that vehicle owners must apply for the IC-chips first and must also install additional wireless capable technology to their vehicles in order to receive information about parking conditions. As a result, there is an initial cost for customers wanting to use car parks managed by these. 1-4.

(5) systems, and it is also very inconvenient for customers who want only occasional parking, or on a once-off basis.. 1.2 Problems of Parking Management Systems According to the factors mentioned above, there are a number of problems which exist in all 3 parking management systems: 1. Ticket keeping: Car owners have to pick up a ticket upon entering the car park, when the time stamp is recorded. Car owners must take good care of the tickets to ensure that information is able to be read off them when required. Furthermore, if the tickets are lost, it is difficult to calculate the parking fee correctly. We can conclude then, that tickets may cause drivers inconvenience, and should therefore not be used in a convenient parking lot. 2. Vacancy display: There is usually a monitor in each main area of a car park to display if there are spaces near that area that are vacant. These monitors usually rely on sensors to detect whether a vehicle is occupying that space, to determine whether it is available or not. However, these results often depend on whether a driver follows the correct directions given. If a driver goes in the wrong direction, the system may produce an incorrect outcome. However, in an ideal car park, this type of mistake should not bede pe nde ntoni ndi vi dua ldr i ve r s ’e r r or s . 3. Inefficient directions: Parking spaces are often close to fully occupied during rush hours, and vacant car parks become very scarce. Inefficient directions being given wa s t e st hedr i ve r ’ st i mea ndi nc r e a s e sa i r pollution inside the car park as vehicles circle around. Therefore, to establish an environmentally-friendly car park, as well as to be more efficient for drivers, it is imperative that the most efficient driving directions be given to drivers. 1-5.

(6) 4. Overnight parking: In order to calculate the parking fee for customers who have lost their tickets, car parks must dispatch someone from patrol to record the number plate of every vehicle in the car park (normally around midnight) so as to ascertain which vehicles are parked overnight. This process, however, can lead to a very heavy workload for staff and could be removed altogether by recording the timestamp of a ve hi c l e ’ sa r r i va la ndde pa r t ur ea ut oma t i c a l l y . 5. Security monitoring: Videotaping systems exist inside the parking lot of most buildings and are streamed directly to monitors, which are set up for management and security staff to oversee the whole area.. Nevertheless, considerable space is needed for the. installation of these monitors and it is bad for the eyesight of the staff.. In addition,. the car park is usually equipped with 24-hour video surveillance to prevent problems at any time.. Thus, plenty of videotapes are needed, or the quality of the video is poor. because of delayed recording. In sum, a good monitoring system should be able to be used to watch over all the places, with fewer "watch dogs" and so as to record without distortion. 6. Safety and disaster handling: A parking management system ideally should be able to identify or detect accidents and disasters such as fire, flood, or earthquake. It should then automatically request assistance from the proper authorities as well as informing t heve hi c l e s ’owne r soft hes i t ua t i on.Thes y s t e ms houl da l s obea bl et or e c omme nd whether it would be safe for owner to retrieve their vehicles, or if it would be best to evacuate. Upon detecting dangerous levels of chemicals such as smoke, poisonous gas or carbon dioxide, the system should automatically trigger alarms to alert customers in the area of the need to evacuate. 7. Parking location reminders: In particularly large car parks, customers often forget the exact location of their parked vehicles and have to go to the added trouble of looking. 1-6.

(7) for a staff member for assistance. This process can be both very time consuming and stressful for the customer. To combat this problem, there should be a simpler system to allow owners to locate their cars quickly and efficiently. 8. Payment: A car park usually needs a large number of staff dedicated to managing the payment process. Although installing automatic payment machines can solve the problem in a standard car park, a larger car park would usually offer many more services, including parking for temporary drivers, for monthly members, for contract users, for the handicapped, for its own employees and for reserved vehicles. The parking rates for each type of service will vary, and this added complexity is not best served by using automatic payment machines. Therefore, a large multi-functional car park has to be able to satisfy the needs of every type of customer, while keeping operating costs to a minimum. 9. Reservations: Drivers usually don’ t know whether parking spaces are available or not until they arrive at the parking lot. If there is not, they either stay and wait, or go to look for another car park. Therefore, a convenient car park should offer enough information for drivers in advance as to whether there are still parking spaces available. Drivers can then find a parking space as soon as they arrive, without any additional inconvenience or time wasted. 10. Vehicle identification: A standard car park often only provides parking space, without any added security measures to combat problems such as stolen vehicles or vehicles which may be employed for criminal activities. The fact that these types of scenario cannot be automatically detected in a standard car park is a concern for public safety. A better car park should be vigilant in preventing these security risks, and should also be useful in assisting police to identify and arrest criminals. 11. Monitoring crime: Car parks are known locations where crime may be committed,. 1-7.

(8) due to the fact that they are little trafficked by pedestrians. Crimes can range from robbery and stealing to much more serious matters, such as fights involving weapons and even explosives. A secure car park should be able to detect if any of the above scenarios occur, and inform the proper authorities automatically. It should also be useful to help identify the criminals afterwards. 12. Storage and proof: A standard car park only records the arrival and departure times of a vehicle on the receipt. A better car park management system should also record the ve hi c l e ’ sma ke ,r e g i s t r a t i onnumbe r ,s t y l e ,color, and the total fee. This could then be used later as proof that a vehicle was present at a particular time. 13. Accident Identification: Accidents often occur in car parks with bad lighting or smaller spaces, and because there are often not many people present to witness the accident, it may be more difficult to find the party at fault afterwards. One common example is when a vehicle is maneuvering in or out of a parking space; it can collide with a neighboring vehicle if the driver misjudges the distance. However, once the car leaves the car park, it then becomes very hard to find the vehicle and its owner. A responsible car park should therefore try to monitor these situations and keep valid evidence of accidents, should further investigation be required. Nowadays, the above-mentioned problems of car parks actually can be improved by utilizing the latest technology. Just by effectively integrating some of the facilities already present in a car park, we can solve many of its current problems. The purpose of this research is to discuss how to introduce new technology to integrate the resources of current car parks and to build up a high-tech intelligent parking system to solve some of the most common problems.. 1-8.

(9) 1.3 Research Aims Due to the disadvantages of present car park systems, as discussed above, we hope to build an Intelligent Car Park System which would be used to achieve greater security for customers and vehicles in the car park as well as allowing staff to manage the car park effectively.. Moreover, customer should be able to park their vehicles in an effective. manner and without any concern f ort he i rownort he i rve hi c l e ’ ss a f e t y . The following issues are strategies that we propose to solve the most common problems in standard car parks. 1. Ticket storage: In the proposed system, video cameras will be used, together with license plate identification software to record every vehicle that enters or exits the car park. In addition, with the timestamps recorded, the system will be able to automatically calculate the payment required without the use of tickets. 2. Vacancy display: The proposed system will include 360 degree omni-directional video cameras and the required image processing technologies to monitor a particular area and to identify vacant spaces. The total number of vacant spaces will then be displayed at the entrance of the car park so as to inform customers before they enter. 3. Improved efficiency in directions: Each intersection of the proposed car park will be equipped with a direction guidance indicator. When a vehicle reaches the intersection, the system will search for the nearest vacancy and direct the driver to that position. 4. Overnight parking: In this proposed system, when a vehicle enters the parking lot, information such as the ve hi c l e ’ snumbe rpl a t e ,a r r i va l time, and the car space location will be automatically recorded. Therefore, additional staff will not be required to record the information of every vehicle manually at night. Instead, they will only have to check the computer system, or print out the required information when needed. This will not only make processes more automatic, but is also advantageous in saving costs 1-9.

(10) which would otherwise be needed to pay for additional personnel. 5. Security monitoring: In the proposed system, a more user-intuitive interface, as well as a better video recording interface, will be implemented to allow staff to monitor the entire car park as they please. When accidents occur, the system will be able to sound an alarm to alert staff of the incident, as well as recording a clear and vivid image without distortion, through the new video recording interface. 6. Safety and disaster handling: Sensors to detect fire, flood, smoke, temperature extremes, and excessive levels of carbon dioxide, will be installed throughout the entire car park. As soon as a sensor detects the possibility of one of these incidents, the system will send an alarm to alert the staff, initiate the appropriate processes and adjust the monitor view to focus on the area in question. If the incident turns out to be a false alarm, staff will be able to easily end the alert. However, in the case of a real emergency, staff can initiate the related systems or disaster-combating equipment straight away, and will also be capable of requesting assistance from the authorities, as well as notifying car owners to drive their cars away. 7. Notification: The information of members (e.g. monthly payers) and broadcasting systems around the car park will also be integrated in this new system. As soon as an accident occurs, staff will be able to use the community broadcasting system nearby to notify car owners to drive their vehicles away. 8. Parking location reminders: In this system, the arrival time, a photo of the vehicle and the parking location of the vehicle are recorded. If the driver is not able to locate their vehicle, they can simply input the arrival time and the plate number into the system, whi c hwi l lt he nc he c kt heda t a ba s ef ort heve hi c l e ’ sl oc a t i on.Amap with directions to the space in question will then be displayed and printed to allow customers to find their vehicles as quickly and efficiently as possible.. 1-10.

(11) 9. Payment: The types of payment included in this system consist of monthly payments as well as the traditional payment processes. Customers with a contract with the car park will also be able to make their payment through the Internet. 10. Reservations: Due to the difficulty of searching for a parking space in rush hour, the proposed system will offer customers the option to reserve a car space. Drivers can check the capacity of the car park beforehand, choose the time required and make a reservation through the website. This will mean drivers no longer have to worry about whether their space will be occupied and then having to look for another parking lot. 11. Vehicle identification: Because the system can identify the number vehicle of every vehicle automatically, every vehicle entering the car park can be filtered and checked, in cooperation with the National Police Administration. Illegal or stolen cars can then be detected, allowing the system to notify the staff and appropriate authorities to deal with the matter. 12. Monitoring crime: The proposed car park will be equipped with a large number of video cameras which are each capable of extracting, analyzing, and judging the recorded images from around the car park. Once a suspicious person or item is detected, the system will launch the video cameras and bring the matter to the attention of staff nearby. In doing so, the aim is to protect the customers and vehicles inside the car park. 13. Storage and proof: In the proposed system, the number plates, as well as the entrance and exit time stamps, and a photo of the vehicle are recorded. Therefore, if an incident occurs in relation to the car park, staff will be able to identify the exact vehicle and extract the required information. 14. Accident identification: When a vehicle enters the car park, special video cameras will be launched to trace the route of the vehicle and guide it to the best vacant space.. 1-11.

(12) Moreover, the driving route is recorded until the vehicle has been parked successfully and the driver leaves.Si mi l a r l y ,t hevi de oc a me r a swi l la l s or e c or dt heve hi c l e ’ se xi t from its space until it leaves the car park. So, if any accident occurs inside the car park while the vehicle is moving, staff will be able to verify it. This can also be useful for accident identifications and for resolution between parties who may otherwise dispute the matter for lack of concrete evidence.. In order to achieve the requirements of the intelligent parking system mentioned above, the research will integrate image processing, computerized monitoring, inductive controlling, transportation, networking, and advanced management technologies. This will allow the existing resources of car parks to be integrated effectively and to offer an automatic and highly efficient management system for staff and management. In this way, drivers can also enjoy a secure environment as well a more convenient and customer-focused service.. 1.4 Organization of This Dissertation The remaining sections of this thesis are organized as follows. In Chapter 2 presents the Intelligent Parking Lot System (IPLS). The three sub systems, License Plate Recognition system, Vehicle Guidance system, and Device Control system, are discussed in detail in Chapters 3, 4, and 5, respectively. Chapter 6 proposes hardware architecture for some programs in the LPRS such that parallel processing can be used in hardware to accelerate the processing of license plate location. Finally, concluding remarks and ideas for future work are given in Chapter 7.. 1-12.

(13) Chapter 2 Intelligent Parking Lot System. In the proposed Intelligent Parking Lot System, the minimum number of omni-directional cameras will be set up in appropriate positions to monitor the entire parking lot. In addition, various sensors, direction indicators, control circuits, and alarm systems will also be set up to assist the operations of the parking lot system. The placement of these facilities is shown in Figure 2.1. Direction Indicator LPR Camera. Flood Detector. Fire Detector. Gas Detector. Omni-Directional Camera Vacancy Display Vehicle LPR Sensor Camera. Gate. Vacancy Display. Surveillance Camera. Fig. 2.1 Facilities inside the parking lot. The operation of the parking areas is as follows: Each gate is initially equipped with various sensors, cameras, and a gate controller. Upon entering the area, the vehicle is detected by the sensors and the Central Management System will launch the appropriate video cameras. The cameras will record the images and other vehicle information such as 2-1.

(14) the license plate. Unless there are no available spaces, the gate will not be closed, and otherwise, the gate will remain open a sl onga st heve hi c l e ’ sl i c e ns epl a t e shave been successfully recognized. This is to reduce the pause time caused when the gates are opened and lowered each time a vehicle approaches. Once inside the car park, the driver will be able to consult the signboard showing available spaces in that particular area. The omni-directional cameras and directional indicators will direct the vehicle to the best available space, whilst at the same time, tracking its route. Finally, standard cameras will be set up to monitor the activities of vehicles and people inside the parking lot. The system will also be capable of sending information out to people (e.g. managers, the police, and the car owners) or launching the alarm system when an abnormal condition is detected. Inside the parking lot, auto payment machines and various alarms and fire-fighting systems will be set up to deal with driver payments and various abnormal conditions. When the vehicle reaches the exit, a sensor at the exit informs the Management Unit to carry out the identification of the license plate and to check whether a payment has been made or not. If the vehicle is trying to leave without paying, the Management Unit will close the gate and, using the interactive voice response system, request that the driver completes payment. Otherwise, if a successful payment is detected, the gate will remain open to allow the vehicles to drive away immediately. As mentioned above, the system is required not only to be able to manage all the vehicles, parking areas and payments efficiently; it also needs to be aware of various incidents that may occur inside the car park. If something unusual is detected, the system should respond to the situation appropriately, such as sending out alert signals or notifying staff of the problem. As for electronic or alarm facilities such as gate switches, alarm systems, and auto-fire-fight systems, an intelligent parking lot system should be. 2-2.

(15) able to automatically control or launch these facilities. Therefore, a complete Intelligent Parking system should be equipped with the functions of surveillance and control, in addition to the key management functions. Our proposed Intelligent Parking Lot System does so by combining the Management Unit, Monitoring Unit, and Control Units together. Their functional structures are shown in Figure 2.2. Management Unit. IPLS Monitoring Unit. Control Unit. Fig. 2.2 Functional structure of IPLS. In order to meet the functional requirements of the parking lot, this research divided the Intelligent Parking Lot System (IPLS) into seven sub-systems: Central Management system (CMS); Payment and Inquiry System (PIS); Network System (NS); Security Surveillance System (SSS); Vehicle Guidance System (VGS); License Plate Recognition System (LPRS); Device Control System (DCS).. We classify these seven sub-systems. into three units according to their functions: Management Unit; Monitoring Unit, and Control Unit (Fig. 2.3). We now introduce the structures, processes, and functions of each of the three units.. 2-3.

(16) PIS. SSS. Monitoring Unit. Management Unit. CMS. NS VGS. LPRS DCS. Control Unit. cameras devices. sensors indicators. Fig. 2.3 System structure.. 2.1 Management Unit The Management Unit deals with problems relating to people, vehicles, payment, and spaces in the parking area. Its framework is shown in Fig. 2.4. For the management of people, information on regular customers, such as names, number plates, addresses, phone numbers and other means of identification will be recorded, as well as the information on the parking lot managers and staff. The purpose of establishing the information on customers is to be able to notify the owners quickly, whether for an emergency in cases such as flooding, or for customer-friendly notices such as pending parking expiration. The establishment of information on the managers is devised to establish the limit of authority of the managers and to be able to effectively manage staff members using this system.. 2-4.

(17) Management Unit. Space. Vehicle. People. Payment Manual. Automatic. Carport. Driveway. Privileged. Member. Customer. Member. Customer. Privileged. Automatic. Contracted. License Number. Image. Time. Network. Empty. Occupied. Public. Handicap. General. VIP. Contracted. Regular. Staffs. Administrators. Vehicle Owners. Fig. 2.4 Management unit framework.. As for vehicle management, we decided to set different arrangements for temporary parking, contract parking, special customers, and some other "problem" vehicles. For temporary parking, we record the entering and leaving times, the fee, and the position of each vehicle. The information may be used for identification or account-checking in the future. For contract parking, information related to the vehicles, such as driver information, license plate, vehicle model, vehicle color, parking deadline, and previous parking records is recorded. Furthermore, one could plan to offer special privileges and services to neighborhood organizations, to attract them to sign up as special customers. Finally, for "problem" vehicles, we plan to cooperate with the police to build up a database of vehicles that are stolen or owned by criminal suspects. Whe n“ pr obl e m”ve hi c l e se nt e rt hepa r ki ng lot, the system will be able to identify the number plate and notify the managers either to act on the situation or to inform the police so as to avoid possible criminal activity. To manage charging, both auto-charging systems and manual toll collection, which 2-5.

(18) is mainly used for temporary parking, will be used. Auto-charging systems allow direct or on-line payment. The former is aimed at temporary parking customers, who can stop at the auto-charging machines, input t he i rve hi c l e ’ snumbe rpl a t eand then pay for their parking directly. Special drivers such as students, attendants, and the handicapped can also input their information at this stage, to receive further discounts. On the other hand, drivers from other organizations (e.g. hospitals, restaurants, theaters) in the surrounding neighborhood of the parking lot can choose to pay online. For these customers, the system will be able to automatically record their parking time, fee, and number plate, so that both the customer and the neighborhood organizations will be able to check their account usage, as well as parking trends. Finally, manual toll collection will be used as a last resort in case of system failure. As for area management, the system can check if each car space is occupied or not. If it is, the number plate of the vehicle will be recorded. Owners and managers can then check the position of the vehicle and get a display of the map of the parking lot, indicating the position of their vehicle. Reservation services will also be available for customers, by informing t hes y s t e m oft heve hi c l e ’ sr e g i stration number and the expected arrival time. During this period, the system will reserve the parking space for the vehicle and guide it to its reserved space upon arrival.. 2.2 Monitoring Unit The Monitoring Unit will be used to monitor people, vehicles, and areas in the parking lot through the Security Surveillance System (SSS), to analyze continuous images captured by the video cameras and to detect any abnormal conditions inside the car park. If an abnormal incident is detected, alarm signals will inform managers and staff of the situation so they can deal with it. The Monitoring Unit shown in Fig. 2.5 is divided into. 2-6.

(19) people, vehicles, and spaces. For monitoring people, the system will distinguish between managers and drivers once they enter the parking lot. If the Monitoring Unit identifies a potentially dangerous object, the system will alert the manager of the situation, but if the system detects criminal or malicious activities in progress such as theft or vandalism, it will notify the manager as well as activating the alarm systems. As for vehicle surveillance, the system will automatically identify the number plate of every car entering the parking lot and lead them to the nearest space using the direction indicators. If anything abnormal occurs, such as a broken down vehicle, illegal parking, or any other type of accident, the system will notify the managers to deal with it either by towing the vehicles or by informing vehicle owners of the situation. For area management, the system will regularly monitor the entire car park to check whether the parking spaces are occupied or not. The information is then updated in the CMS, so that the system will be able to lead vehicles to the available spaces later on. Area Management will also manage the unoccupied spaces as well.. Monitoring Unit. People. Space. Vehicle. Carport. Driveway. Abnormal. Normal. Damage. Steal. Violation. Incident. Out of order. Driving. Parking. Vehicle. Facilities. Vehicle. Facilities. Fig. 2.5 Monitoring unit framework.. 2-7.

(20) 2.3 Control Unit The Control Unit uses micro-computer control and detecting technologies to manipulate the facilities in the parking lot. It can be divided into three sub-systems: Gate, Sensors, and Signals, as shown in Fig. 2.6. The gate system is in charge of detecting vehicles, as well as the switch control of the gate. The former is to detect if there are any vehicles entering the parking lot. If so, the system will transfer the information to the Central Management System to record the image and the number plate.. The latter wait. for the identification outcome from the LPRS, and then determines whether the gate should be opened for the vehicle to enter or remain closed to keep the vehicle out.. Control Unit. Gates. Sensors. Signals. Direction. Number. Vehicle. Temperature. Flood. Air Pollution. Image. Exit. Entrance. Recognition. Security Surveillance. Vehicle. Fig. 2.6 Control unit framework.. The Security Surveillance System (SSS) is designed to detect occasions such as floods and the outbreak of fire, and to raise the alarm. If the possibility of a flood or fire is detected, the SSS will immediately send an alarm signal to the CMS to notify the managers. If the accidents are confirmed, the alarm will activate the fire-fighting equipment and the notification system, which will in turn inform the local fire department as well as the car owners.. 2-8.

(21) The Indicator system consists of high-luminance LEDs, which are used to display the number of available parking spaces within each area. The system will also operate the guidance indicators to assist vehicles in navigating inside the car park as well as those who are departing. In this way, the amount of time needed to search for a vacant space is greatly reduced, which in turn will also improve air quality and the environmental impact of the parking lot.. 2-9.

(22) Chapter 3 License Plate Recognition System. Automatic license plate recognition (LPR) plays an important role in numerous applications and a number of techniques have been proposed. However, most of them worked under restricted conditions, such as fixed illumination, limited vehicle speed, designated routes, and stationary backgrounds. In this study, as few constraints as possible on the working environment are considered. The proposed LPR technique consists of two main modules: a license plate locating module and a license number identification module. The former characterized by fuzzy disciplines attempts to extract license plates from an input image, while the latter conceptualized in terms of neural subjects aims to identify the number present in a license plate.. 3.1 Introduction Automatic license plate recognition (LPR) plays an important role in numerous applications such as unattended parking lots [Sir 98, Yun 35], security control of restricted areas [Dra 97], traffic law enforcement [Dav 90, Yam 99], congestion pricing [Cow 95], and automatic toll collection [Lot 90]. Due to different working environments, LPR techniques vary from application to application. Most previous works have in some way restricted their working conditions [Emi 01], such as limiting them to indoor scenes, stationary backgrounds [Sal 99], fixed illumination [Dav 90], prescribed driveways [Miy. 3-1.

(23) 91, Par 98], limited vehicle speeds [Ado 98], or designated ranges of the distance between camera and vehicle [Nai 00]. The aim of this study is to lessen many of these restrictions. Of the various working conditions, outdoor scenes and non-stationary backgrounds may be the two factors that most influence the quality of scene images acquired and in turn the complexity of the techniques needed. In an outdoor environment, illumination not only changes slowly as daytime progresses, but may change rapidly due to changing weather conditions and passing objects (e.g., cars, airplanes, clouds, and overpasses). In addition, pointable cameras create dynamic scenes when they move, pan or zoom. A dynamic scene image may contain multiple license plates or no license plate at all. Moreover, when they do appear in an image, license plates may have arbitrary sizes, orientations and positions. And, if complex backgrounds are involved, detecting license plates can become quite a challenge. Typically, an LPR process consists of two main stages: 1) locating license plates and 2) identifying license numbers. In the first stage, license plate candidates are determined based on the features of license plates. Features commonly employed have been derived from the license plate format and the alphanumeric characters constituting license numbers. The features regarding license plate format include shape, symmetry [Kim 01], height-to-width ratio [Nai 00, Nij 95], color [Kim 96, Nij 95], texture of grayness [Bru 98, Nij 95], spatial frequency [Par 98], and variance of intensity values [Dra 97, Gao 00]. Character features include line [Yu 00], blob [Hon 01], the sign transition of gradient magnitudes, the aspect ratio of characters [Her 97], the distribution of intervals between characters [Poo 95], and the alignment of characters [Soh 94]. In reality, a small set of robust, reliable, and easy-to-detect object features would be adequate. The license plate candidates determined in the locating stage are examined in the license number identification stage. There are two major tasks involved in the. 3-2.

(24) identification stage, character separation and character recognition. Character separation has in the past been accomplished by such techniques as projection [Heg 98, Sal 99], morphology [Bru 98, Gao 00, Poo 95] relaxation labeling, connected components [Nij 95], and blob coloring. Every technique has its own advantages and disadvantages. Since the projection method assumes the orientation of a license plate is known and the morphology method requires knowing the sizes of characters, these two approaches are not appropriate for our application because of their required assumptions. Relaxation labeling is by nature iterative and often time consuming. In this study, a hybrid of connected components and blob coloring techniques is considered for character separation. There have been a large number of character recognition techniques reported. They include genetic algorithms [Kim 96], artificial neural networks [Bru 98, Kim 00, Par 98], fuzzy c-means [Nij 95], support vector machine [Kim 00], Markov processes [Gui 97], and finite automata [Ado 98]. These methods can be broadly classified into iterative and noniterative approaches. There is a tradeoff between these two groups of approaches; iterative methods achieve better accuracy, but at the cost of increased time complexity. In this study, we pay more attention to accuracy than time complexity whenever a choice has to be made between them. For this, we developed our own character recognition technique, which is based on the disciplines of both artificial neural networks and mechanics. The rest of this chapter is organized as follows. In Section 3.2, the types of license plates to be considered are described, followed by the fundamental idea of the proposed LPR technique. The two primary stages of the proposed technique, license plate location and license number identification, are discussed in detail in Sections 3.3 and 3.4, respectively. Experimental results are presented in Section 3.5. Concluding remarks and. 3-3.

(25) ideas for future work are given in Section 3.6.. 3.2 LPR In this section, the styles of license plate that are considered in this study are discussed, followed by a brief description of the proposed LPR process. Table 3.1 shows assorted styles of license plates found on vehicles in Taiwan. Each style is associated with a particular class of vehicle. The classes include private automobile, taxi, tour bus, truck, and government vehicles. Other categories of vehicles, such as diplomatic cars and military vehicles, are not addressed since they are rarely seen. Styles of license plates can easily be distinguished based on two attributes: 1) the combination of colors used and 2) the compositional semantics of license numbers.. Table 3.1 The kinds and specifications of license plates. Vehicle type. Plate color Character color. Example. Private car. White. Black. E1－2345. Taxi. White. Red. E1－234. Tour bus. Red. While. E1－234. Truck. Green. While. E1－234. Government vehicles. White. Green. E1－234. As shown in Table 3.1, each style has a different foreground and/or background color. However, in all only four distinct colors (white, black, red, and green) are utilized in these license plates. We shall pay attention to these four colors when searching for license plates in an input image. The compositional semantics of license numbers provides additional information for differentiating styles of license plates. As can be seen in Table 3.1, every license number is composed of two parts separated by a hyphen (e.g.,. 3-4.

(26) E1-2345). The first part consists of two characters, one of which must be an alphabetical character (e.g., E1, 2F, and EF). The second part may contain four (e.g., 2345) or three (e.g., 234) numerals, the former being used only on private automobiles, and the latter being used on the other vehicle classes. Fig. 3.1 shows the proposed LPR process. We assume that the process is incorporated in an event detection system, e.g., a vehicle detector or a traffic law enforcement system. Once the system detects an event, the camera along with the system is activated. The image acquired by the camera is then sent to the LPR process, in which potential license plates are extracted from the image. If no license plate is found, the process returns to await another input image. However, oftentimes multiple license plate candidates are detected. They are closely examined at the license number identification stage. There are two essential tasks involved in this stage, character segmentation and recognition. These two tasks are alternatively invoked in order to achieve optimal results for both segmentation and recognition. The characters recovered from a license plate candidate at this stage are next verified at the confirmation stage. The group of characters will be deemed to form a valid license number if it agrees with the compositional semantics of license numbers mentioned earlier. Both the valid license number and the associated vehicle class will be returned by the LPR process. The identification and confirmation stages repeat for all of the license plate candidates. Afterwards, the process returns to await another input image. In Sections 3.3 and 3.4, we look at the details of the license plate locating module and the license number identification module.. 3-5.

(27) Fig. 3.1 Diagram of the proposed LPR process.. 3.3 License Plate Locating Module 3.3.1 Basic Concepts A flowchart for the license plate locating module is shown in Fig. 3.2. The input to this module is an RGB color image. Recall that only four colors (white, black, red, and green) are utilized in the license plates that we consider. Note also that there are many edges, which are in close mutual proximity and are dispersed in a repetitive manner, contained in a license plate. The above observations motivate us to develop a color edge detector. The edge detector is sensitive to only three kinds of edges, black-white, red-white, and green-white (see the last column of Table 3.1). By ignoring other types of edges in an image, very few edges due to objects other than license plates are detected, even when the image background is very cluttered. Let E denote the edge map computed from the input image using the color edge detector.. 3-6.

(28) Fig. 3.2 Flowchart for the license plate locating module.. Next, the RGB space of the input color image is transformed into the HSI space. Let (R, G, B) and (H, S, I) denote the (red, green, blue) and (hue, saturation, intensity) values of an image pixel, respectively. The transform from (R, G, B) to (H, S, I) [Cas 96] is (r g b) I 3 min(r , g , b) S 1  I   (r g ) (r b) H cos 1  2 1/ 2  2[(r g ) (r b)( g b)] . (3.1). where r=R/255, g=G/255, and b=B/255. There are a number of intriguing characteristics associated with the HSI color model which are useful for our application, including the invariance of hue to both illumination and shading, and the invariance of saturation to both viewing direction and surface orientation. Let H, S, and I be the maps preserving the hue, saturation, and intensity components of the transformed image, respectively. It is inevitable that maps E, H, S, and I are less than perfect in view of noise, measurement error, and imperfect processing. In order to compensate for this drawback, 3-7.

(29) we appeal to the soft computing techniques rooted in fuzzy (for license plate location) and , S, I, and E  be the neural (for license number identification) disciplines. Let H fuzzy versions of H, S, I, and E. The entries in the fuzzy maps represent the degrees of belonging to a license plate. A two-stage fuzzy aggregator is introduced to integrate the , S, and I are integrated. The resulting map maps. In the first stage, fuzzy maps H  in the second stage leading to a single map, denoted M . The next combines with E reason of using the two-stage aggregator is because the intrinsic characteristics (related to , S, and I are different from that (related to edge magnitude) of E . color) of H , interesting regions are located in the input image, which Afterwards, based on map M have locally maximal mvalues. License plate candidates are then determined to be those interesting areas whose sizes are large enough.. 3.3.2 Color Edge Detection The color edge detector focuses on only three kinds of edges (i.e., black-white, red-white and green-white edges). Consider a black-white edge, and suppose that the input RGB color image has been normalized into an rgb image. Ideally, the (r, g, b) values of a white pixel and a black pixel should be (1, 1, 1) and (0, 0, 0), respectively. Their differences ( r , g , b ) are either (1, 1, 1) or (-1, -1, -1), so all the components of the difference vector between a white and a black pixel will have the same sign. This property is considerably stable under environmental variations. A black-white edge pixel is then defined based on this property as follows. An image pixel is regarded as a black-white edge point if all of the signs of components of the difference vector between the pixel and one of its neighbors are the same, i.e., sign(ri )=sign(gi )=sign(bi ) , i N , where N is the set of neighbors of the image pixel. We also store its edge. 3-8.

(30) magnitude defined as min ri , g i , bi . Edge magnitudes will be exploited later to derive fuzzy edge maps. In a similar way, an image pixel is considered to be a red-white edge point if its difference vector ( ri , gi , bi ) for some i N satisfies the following conditions: 1) sign(ri )=sign(gi )=sign(bi ) and 2) ri gi and ri bi . The magnitude of the edge pixel is defined as min g i , bi . Finally, an image pixel is regarded to be a green-white edge pixel if for some , 1) sign(ri )=sign(gi )=sign(bi ) and 2). gi ri and gi bi . Its edge magnitude is determined by min ri , bi . Image pixels, which are not edge points, are given zero edge magnitudes.. 3.3.3 Fuzzy Maps The basic idea of generating a fuzzy map from a given map (e.g., H, S, I, or E) is as follows. Since every map encodes some characteristic about the scene, the entry of any cell in the map expresses the degree of the cell possessing the property. In order to highlight the cells corresponding to the objects of interest (e.g., license plates), we assign large entries to those cells that are compatible with the known characteristics of the objects. Such large entries indicate a high degree of existence of an interesting object. We call the resultant map the characteristic map of the original map. Since the input data (both the given map and the object characteristics) are not perfect, uncertainties should be taken into account during the computation of the characteristic map. Fuzzy sets have been known to provide an elegant tool for modeling uncertainties [Kel 94, Kli 95, Per 02]. In this study, we introduce fuzzinesses into the entries of the characteristic map and refer to the result as the fuzzy map. There are several ways to realize fuzziness. Wede f i neag e ne r a l i z e df uz z ys e t ,t e r me d“ l i keal i c e ns epl a t e , ” 3-9.

(31) on the respective sets of hue, saturation, intensity, and edge magnitude. Each of the four sets serves as a universal set of the fuzzy set. Map: Consider the universal set of hue values. Suppose that the object of interest 1) H has a color C whose corresponding hue value is hc. Given an entry in map H, say h, the membership degree, μc, of the entry belonging to fuzzy set “ l i ket heobj e c t ”c a nbe written. c (h) exp(a h hc ) where a is a positive constant. If the given entry h is equal to that of the interesting object hc, then the degree of membership is 1. As the difference between the hues increases, the degree of membership decreases to an asymptotic value of 0. Recall that there are four colors (black, white, red and green) utilized in the license plates that are of interest. Let hr and hg be the hue values for red and green, respectively. Note that the hues of achromatic colors (i.e., all levels of grey, including black and white) are not defined since the denominator of the equation for hue in (3.1) is zero. Therefore, we will highlight red and green, but not white and black based on map H. The membership function of fuzzy is finally defined as map H. H(h) u ( r (h), g (h)). (3.2). where u can be any fuzzy union operator (any t-conorm function).  can only express the colors red and green, we need other 2) SMap: Since fuzzy map H means to handle black and white. According to the S in (3.1), all achromatic colors have the same saturation S. In addition, this value is smaller than that of any chromatic color. Based on these two facts, we generate a fuzzy map S from map S for distinguishing between chromatic and achromatic colors. The membership function of S is defined as. S(h) exp(as ) . 3-10. (3.3).

(32) This states that the smaller a given saturation value the more likely that it comes from some achromatic color. 3) I Map: While chromatic and achromatic colors can be separated from each other based on their saturation values, black and white have to be further differentiated from other achromatic colors. For this, we count on the intensity map I. Since the intensity values of black and white correspond to the two extreme values on the intensity axis of the HSI coordinate system, the following function emphasizes the colors with intensity values close to the two extremes f (i ) 1 exp[ a (i 0.5)] .. This assumes that the working environment has an average intensity of 0.5. However, both black and white will be distorted under some circumstances. For example, a white color may appear grey in a dark environment, as will a black color in a bright environment. To compensate for such distortion, the constant 0.5 in the above equation may be replaced with the average value, i , of map I. We then define the membership function of fuzzy map Ias. I(i ) 1 exp(a(i i )) .. (3.4).  Map: Based on fuzzy maps H , S, and I, image areas with black, white, red, or 4) E green colors can be distinguished. However, a large portion of these areas has nothing to do with license plates. Edge maps play a crucial role in discriminating against irrelevant regions. Since there are many close-by edges in a license plate and distributed in a repetitive manner, an image pixel whose neighbors possess large edge magnitudes will have a high possibility that it belongs to a license plate. Hence, we define the membership  as function of fuzzy edge map E. E(e p ) ek exp(ad pk ). (3.5). kN p. 3-11.

(33) where Np is the horizontal neighborhood of the image pixel p under consideration, ek is the edge magnitude of pixel k in Np, and dpk is the Euclidean distance between pixels p and k . In the above function we do not care about the edge magnitude ep of pixel p itself.. 3.3.4 Fuzzy Aggregation Each fuzzy map provides information for locating license plates in the input image. There are two ways to draw a conclusion from a set of maps. In the first, intermediate decisions are made on the basis of individual maps and a final conclusion is drawn from the intermediate decisions. In the second, multiple maps are first integrated into a single map, and a final conclusion is then drawn from the integrated map. Since the first method involves both numerical computations and symbolic decisions, we prefer the second approach, which includes only numerical computations. Following the second approach, , S, I, and E are integrated into a single map, M , with decisions being fuzzy maps H . A two-stage fuzzy aggregator is introduced for this purpose. made on the basis of M , S, and Iare integrated cell by In the first stage of the aggregator, fuzzy maps H cell. Let h, s,and i be the entries of the corresponding cells in fuzzy maps. , H. S,and  I . The aggregator integrates the entries by gu ( wh h , ws s , wi i ). (3.6). where u is a fuzzy union operator and wh, ws, and wi are weights reflecting the relative , S, and I. The weights are described degrees of importance among fuzzy maps H next. Recall that in the definition of a fuzzy map a large entry indicates a high degree of possibility that the entry belongs to a license plate. However, if the majority of cells having small variations in the entry (i.e., having nearly uniform distribution of entries) are. 3-12.

(34) in a fuzzy map, the usefulness of the map for detecting license plates deteriorates. To see this, consider a picture taken in the evening or on a rainy day. On the whole, the picture will look dim. The overall intensities of image pixels tend to be small, which in turn leads to large saturation values throughout the picture [see (3.1)]. Both the intensity and saturation maps contribute little to locating license plates because entries are comparable. In all fuzzy maps, it is desirable that a relatively small portion of a map possesses large entries, while the remaining areas contain small values. Such a map will be given a large  weight to reflect a high degree of importance of the map. Let A a ij  be any fuzzy . map of size M by N. Its weight (or degree of importance) is then determined by M. N. b i 1 j =1. wa 1 . ij. MN. (3.7). where. 1, if a  ij t bij  0, otherwise    and a in which threshold t (a are the maximum and max amin ) / 2 and amax min minimum values in A. , S, and I, at the second stage the resulting map, After combining fuzzy maps H. , and fuzzy map E   are linearly combined into M say G mby mwg gwe e. (3.8). and E , which are determined similar to where wg and we are the weights of maps G (3.7).. 3-13.

(35) 3.4 License Number Identification Module 3.4.1 Fundamental Idea. Fig. 3.3 Flowchart for the license number identification module.. Fig. 3.3 gives the flowchart for the identification module. There are two major components constituting the module, preprocessing and recognition. The preprocessing component consists of three tasks, binarization, connected component labelling, and noise removal, which are arranged in sequence. The recognition component is composed of two main procedures, character segmentation and recognition. To obtain optimal results for both the procedures, they are alternatively invoked. Since the camera may be rolled and/or pitched with respect to license plates, it is desirable that their images be transformed to a predefined size and orientation before performing license number identification. However, without information about relationships between the camera and working environments, the transformations can only be conducted blindly or by trial-and-error. In the proposed method since the transformation step is omitted, it is inevitable that difficulties in the subsequent steps will 3-14.

(36) increase. Considering a license plate candidate, it is first binarized. Since some information will somehow be lost during binarization, a variable thresholding technique previously proposed by Nakagawa and Rosenfeld [Nak 79] is employed. The technique determines a local optimal threshold value for each image pixel so as to avoid the problem originating from nonuniform illumination. Although variable thresholding cannot completely compensate for the information loss mentioned above, it at least preserves information that may be lost when using a constant binarization method. There are two purposes for the binarization step: highlighting characters and suppressing background. However, both desired (e.g., characters) and undesired (e.g., noise and borders of vehicle plates) image areas often appear during binarization. In order to eliminate undesired image areas, a connected component algorithm is first applied to the binarized plate candidate. The aspect ratios of connected components are then calculated. The components whose aspect ratios are outside a prescribed range are deleted. Then an alignment of the remaining components is derived by applying the Hough transform to the centers of gravity of components. The components disagreeing with the alignment are removed. If the number of remaining components is still larger than a prescribed number (eight in practice), connected components are deleted one at a time starting with the smallest. Here, we choose eight as the prescribed number because a license number consists of five or six characters and characters may be broken. The removal process continues until either of two conditions is satisfied. Either the number of remaining components equals the prescribed number, or a dramatic change in size from the previously removed component to the current one under consideration is encountered. We assume that noise components are much smaller than characters. The above procedure does not guarantee that each of the surviving components will. 3-15.

(37) correspond to an individual character. A component may be due to noise, an incomplete character, a distorted character, or characters that appear to touch. To distinguish them, we utilize attributes of license plates, including the aspect ratios of individual characters, the regular intervals between characters, and the number of characters constituting license numbers. We refer to these attributes collectively as the structural constraints of license plates. We also introduce the operators of delete, merge, split and recover into the character segmentation procedure. Note that characters may be missed during license plate location and binarization. The recover operator is introduced to retrieve missing characters. During processing the segmentation procedure applies the first three operators (delete, merge, and split) to the set of surviving components in an attempt to determine if a component satisfies the structural constraints of license plates. If such a component can be determined, the character recognition procedure is invoked to identify a character from the component. The above process repeats until no character can be extracted from the set of surviving components. Thereafter, if the number of extracted characters is less than the number of characters in license numbers, the recover operator starts at the outermost characters of those detected and searches for characters along the alignment of the known characters. The search continues until no character can be retrieved within an extent determined by the average width of characters as well as intervals between characters. Next, the collection of identified characters is verified in the confirmation stage, where the compositional semantics of license numbers plays an important role. The set of characters will be deemed to form a valid license number if it agrees with the compositional semantics.. 3-16.

(38) 3.4.2 Optical Character Recognition In this subsection we discuss the character recognition procedure. Since, as already mentioned, license plates may be bent and/or tilted with respect to the camera, characters extracted from such license plates may be deformed. Furthermore, input characters may be noisy, broken or incomplete. Character recognition techniques should be able to tolerate these defects. In this study, we develop our own character recognition approach to suit our particular application. The proposed approach consists of three steps: character categorization, topological sorting, and self-organizing (SO) recognition. In the first step, the input character is distinguished as numerical or alphabetical. This is easily accomplished by referring to the compositional semantics of license numbers. In the next step, the topological features of the input character are computed and are compared with those of prestored character templates. Compatible templates will form a test set, in which the character template that best matches the input character is determined. The template test is performed by a SO character recognition procedure. 1) Topological Classification: The topological features of characters utilized in this study include the number of holes, endpoints, three-way nodes, and four-way nodes (see Fig. 3.4 for their definitions). These features are invariant to spatial transformations (including rotation, translation and scale change). Moreover, these features, which are qualitative in nature, can be easily and reliably detected compared to quantitative features. However, input characters are usually imperfect; extra or missing features may occur. The following rule is employed for topological sorting. A character template is compatible with a given character whenever 1) their difference in the numbers of holes is within the range [1, -1] and 2) their difference between the numbers of nodes of any type is within the range [2, -2]. Here, a smaller range is given to the hole feature because it is generally more reliable in detection than nodes. In our experiments no more than three out of ten numerical. 3-17.

(39) character templates and six out of 26 alphabetical character templates have passed topological sorting for any given character. This has greatly reduced the number of templates in the test set and hence the time complexity for character recognition.. (a) end-point. (b) three-way node. (c) four-way node. Fig. 3.4 Nodal types.. 2) Template Test: The templates in the test set are matched against the input character and the best match is determined. The template test is primarily accomplished using a SO character recognition approach, whichi sba s e donKohone n’ sSOneural network [Koh 89]. The idea behind the proposed technique is as follows. Given an unknown character and a character template, the input character is encoded in terms of synaptic weights in the between-layer links of the neural network. The character template here serves as a stimulus, which repeatedly innervates the neural network, causing the synaptic weights of the neural network to gradually change. This process continues until the weights stabilize. We sum up the changes of synaptic weights during processing. The total change in weight in a sense reflects the level of dissimilarity between the unknown character and the character template. Let C={c1,…,cL} be the set of character templates surviving from the topological sorting of an unknown input character. Let d1,…,dL denote the computed dissimilarities between the unknown character and the character templates. It is natural that the character template having the smallest dissimilarity with the unknown character is taken to be the class to which the unknown character belongs. a) SO Neural Model: In this subsection, we brief the key components of the 3-18.

(40) Kohonen SO neural model, which will be used in the later practical implementation. Referring to Fig. 3.5, the underlying configuration of the SO neural network consists of two layers, an input layer and an SO layer.. Fig. 3.5 Kohonen SO neural model.. Let wij be the weight of the link between SO neuron ni and input neuron nj. The weight vector for ni is wi=(wi1, wi2,…, wim), where m is the number of input neurons. Let v=(v1, v2,…, vm) denote an external stimulus. The input to ni due to the stimulus is m. I is wi  v wik vk. (3.9). k 1. The lateral interaction among SO neurons is characterized by a “ Me xi c a n-ha t ”f unc t i on [Mar 82], denoted h(r), where r represents a position vector. Let uik be the weight of the connection between SO neurons ni and nk located at ri and rk, respectively. The input to ni due to lateral interaction is I il . m. au. nk N , k i. k. ik. h(rk ri ). (3.10). where N is the set of SO neurons and ak=Ψ(netk) is the activation of nk, in which netk to be defined is the net input to nk and Ψ is the output function defined by a sigmoid function. A leakage term e(a), which dissipates activations of SO neurons once a stimulus has been removed, is introduced for every SO neuron. The net input to ni then sums the inputs 3-19.

(41) from the stimulus, lateral interaction, and leakage neti I is I il e(ai ). (3.11). During competition among SO neurons, the winner nc is determined by. netc max1 neti . Next, the winner together with its neighbors, say set Nc, engage in i n a group learning process. During this process a neuron close to the winner will gain a high rate of learning while a neuron located far from the winner will have a low rate of learning. The rate of learning is governed by the Gaussian function g. The learning rule for the neurons in Nc is then defined as wk (v wk ) g (rk rc ),. k Nc .. (3.12). Finally, the updating equation for SO neuron nk is. wkt 1 wkt (t )wkt ,. k N c. (3.13). where ρ (t) is the step size, which decreases monotonically with increasing t. b) Practical Implementation: To begin, we group characters into three categories, referred to as 0-hole, 1-hole, and 2-hole, according to the number of holes contained in the characters. Each category has its own associated SO neural network, which contains 40 SO neurons and two input neurons. The difference among the three neural networks is primarily in their configurations of SO layer (see Fig. 3.6).. (a). (b). (c). Fig. 3.6 SO layers: (a) 0-hole, (b) 1-hole, and (c) 2-hole SO layers.. Referring to the example shown in Fig. 3.7, suppose that we are given an unknown c ha r a c t e r( “ C”i nt hi se xa mpl e ) .Thec ha r a c t e ri snor ma l i z e di ns i z e( 16by16pi xe l s )i n. 3-20.

(42) order to be consistent with the character templates. The number of holes in the character is computed. Here, we always choose the neural network according to the computed number regardless of whether the computed number is the true number of holes for that character. Since the inputc ha r a c t e r( “ C” )ha snohol e ,t hene ur a lne t wor kwi t ht he0-hole SO layer is chosen. Next, the contour and its length of the unknown character are found. The length is divided into 40 approximately equally spaced intervals. Starting at any arbitrary point along the contour, the two dimensional (2-D) position vectors, i.e., the (row, column) coordinates, of the 40 interval boundaries are extracted. See the right side of Fig. 3.7. We choose 40 points because they are about half the average number of contour points in the character templates. The position vectors of the 40 contour points are then assigned, one by one in the order of extraction, to the weight vectors of the 40 SO neurons of the chosen neural network. Note that since all the configurations of SO layer of the three neural networks are symmetrical which SO neuron should be assigned first is not important.. Fig. 3.7 Example of SO character recognition. Consider a 2-D space with axes corresponding to the two components of weight vectors of SO neurons. The weight vector of each SO neuron can be represented as a point in the space. This space is referred to as the weight space of the neural network.. 3-21.

(43) Since the weight vectors of SO neurons are set to the position vectors of contour points of the input character, the contour will be recreated in the weight space when we represent the weight vectors of SO neurons as points in the weight space. See the picture on the right hand side of Fig. 3.7. Suppos et ha tat e mpl a t e( “ L”i nt hee xa mpl e )c hos e nf r om t hetest set for the input character is to be matched against the character. Rather than the entire template, just its contour serves as the stimulating pattern, which repeatedly innervates the neural network until its synaptic weights stabilize. In our implementation the contour points are fed into the input layer of neural network one at a time. Consider an input contour stimulus point v. The SO neurons of the network compete for the stimulus. The winner nc is determined by nc arg(max1 v}) , where wi’ sa r et hewe i g htve c t or soft he40SO ne ur ons . i 40 {wi  The winner and its first- and second-order neighbors, call them set nc, join in the following learning process wk d k g (rk rc ),. k N c. (3.14). Note that (v-wk) in (3.12) has been replaced by dk in (3.14). The dk is computed as follows. We use a model, called the spring model, taken from [Che 96], in which SO neurons are assumed being connected with springs. The elastic spring coefficients are simulated with synaptic weights between neurons. Referring to Fig. 3.8, we denote the SO neurons with their weight vectors. Weight vectors wk-1, wk, and wk+1 represent SO neurons nk-1, nk, and nk+1, respectively, where nk is any learner in Nc and nk-1 and nk+1 are the two first-order neighbors of nk. The learner is connected to its two neighbors with the springs, whose coefficients are uk-1,k and uk,k+1. The point stimulus is denoted v in the figure.. 3-22.

(44) Fig. 3.8 Spring model.. The v here serves as an attractive source, which during the learning process attempts to pull wk toward it with force fv. However, the springs exerting forces fk-1 and fk+1 on wk try to pull wk toward its neighbors wk-1 and wk+1. In addition, there is a damping force, fd (not shown in the figure), for dissipating the energy in the spring model so that the entire system will eventually converge to an equilibrium state with an external force fe, i.e. f v f k 1 f k 1 f d fe. (3.15). where. ka v wk fv  2 ( v wk ) v wk f k 1 uk 1,k ( wk 1 wk l ). wk 1 wk wk 1 wk. f k 1 uk ,k 1 ( wk wk 1 l ). wk wk 1 wk wk 1. f d kd f v f k 1 f k 1. 1/ 2. f v f k 1 f k 1 f v f k 1 f k 1. (3.16). in which ka is the gravitational coefficient, kd is the damping coefficient, l is the natural length of the springs, andεis a small value to prevent fv from becoming infinite as wk approaches v. Substituting the forces in (3.16) into (3.15) for fe, the displacement of neuron nk is f d k v0 k  e t 2 . m. (3.17). 3-23.

(45) For simplicity, we assume that the initial velocity of neuron nk, v0k, is zero, the neural mass m is one, and the time interval Δt is one. The result is d k f e f v f k 1 f k 1 f d .. (3.18). Note that the calculated displacement dk has to be modified by the degree of learning of neuron nk, g(rk - rc), and a learning step ρ (t). The actual displacement of neuron nk in the weight space is ρ (t) dk g(rk - rc). Accumulating the displacements of all the neurons in Nc accomplishes the innervation of point stimulus v. Repeating the above process for all point stimuli of the input stimulus pattern completes one iteration for the pattern. Iterating continues until no significant displacement of SO neuron is observed (i.e., stabilized). The total displacement of SO neurons serves as a measure of dissimilarity between the unknown character and the character template. Fig. 3.9 illustrates some intermediate results of s ha pe t r a ns f or ma t i on f r om “ C” t o“ L” dur i ng i t e r a t i on.In this example the total displacement of neurons amounts to 147 pixels. Empirically, displacements have been distributed over the interval [23,67] when correct templates were chosen for testing.. (a) (b) (c) (d) Fig. 3.9 Some intermediate results of shape transformation. (a) 1st iteration. (b) 5th iteration. (c) 10th iteration. (d) 42nd iteration.. c) Remarks: The proposed character recognition approach has difficulty distinguishing character pairs (8, B) and (O, D) especially when they are distorted. To overcome this, we predefine an ambiguity set containing the characters 0, 8, B and D. For each character in the set, the nonambiguous parts of the character are specified (see Fig. 3-24.

(46) 3.10). During character recognition, once an unknown character is classed as one of the characters in the ambiguity set, an additional minor comparison between the unknown character and the classed character is performed. The comparison focuses on only the nonambiguous parts of the character.. Fig. 3.10 Distinguishing parts of ambiguous characters.. Our character recognition method gives different measurements of dissimilarity for the same character with different tilt angles with respect to the camera. Currently, this issue has not troubled us because the characters extracted from the images of license plates are all in a nearly upright position. But, we may improve our algorithm to deal with this by introducing a normalization step to transform license plates into a prescribed orientation prior to license number identification.. 3.5 Experimental Results Car. c1 b1 1 step 1 step. a1. b2. a2. c2. license plate 2 steps away. Viewpoint a1. 3 steps away. 5 steps away c3. b3. a3 (a). Viewpoint c3. (b). Fig. 3.11 (a) Nine different viewpoints for the first group of images.The length of one step is about 60 cm. (b) Two images of a car taken from viewpoints a1 and c3. 3-25.

(47) Two groups of images have been collected for our experiments. The first contains 639 images (640 by 480 pixels) taken from 71 cars of different classes. For each car, nine images were acquired from fixed viewpoints whose positions are illustrated in Fig. 3.11(a). Fig. 3.11(b) shows two images of one car taken from viewpoints a1 and c3. The experimental results with the first group of images are summarized in Table 3.2. In this table columns correspond to viewpoints, rows to the classes of vehicle (or the types of license plate), and the entries are the number of correctly located license plates. The percent of correctly located license plates (the success rate) is given in the bottom row of the table. The success rates for viewpoints a1, a2, and a3 (i.e., straight on) are 100%, independent of the type of license plate and viewing distance. However, as the viewing angle increases the success rate declines. In the worst case, viewpoints c1, c2, and c3, the success rates are 97.2%, 98.6%, and 95.8%, respectively. The overall average success rate with the first group of images is 98.8%. Table 3.2 Experimental results with the first group of images Viewpoint #vehicles. A1. A2. A3. B1. B2. B3. C1. C2. C3. Private automobile 30 30 Taxi 20 20 Tour bus 7 7 Truck 8 8 Government vehicle 6 6 Success rate(%) 100 100 Average success rate(%): 98.8. 30 20 7 8 6 100. 30 19 7 8 6 98.6. 30 20 7 8 6 100. 30 20 7 7 6 98.6. 30 19 6 8 6 97.2. 30 19 7 8 6 98.6. 29 20 7 6 6 95.8. Vehicle class. The second group contains 449 images (768 by 512 pixels), some of which are shown in Fig. 3.12. The images are taken from (a) complex scenes, in which several objects look like license plates, (b) various environments (street, roadside and parking lot), (c) different illumination (dawn, sunshine, rain, back lighting, shadow, and nighttime), and (d) damaged license plates (such as being bent). In these images, all the license plates. 3-26.