In this chapter, we describe the main ideas and techniques of the proposed methods for used in omni-vision systems.
2.1 A Modified Unifying Model for Omni-cameras
A new modified camera model for omni-cameras, including single viewpoint catadioptric omni-directional cameras, fisheye-lens cameras, etc., is proposed in this study. This new camera model is modified from the unifying omni-camera model proposed by Geyer and Daniilidis [44], and it is with fewer camera parameters than the original unifying model. In the proposed modified model, we investigated an important invariant property, so that one of the parameters in the original unifying omni-camera model can be eliminated by replacing it with an optimal approximated value, and in the meantime preserve an important property regarding to space line detection. Then, according to this invariant property, we design a series of experiments to find an optimal value to approximate one of the parameters in the original unifying model proposed by Geyer and Daniilidis. Furthermore, since the proposed modified model has fewer parameters than the original one, we also proposed a new calibration model using only one straight line in the 3D world, without knowing its position, direction, or length.
To sum up, we investigate a new invariant property regarding to the projections of the straight lines by omni-directional cameras, and accordingly proposed a new modified unifying omni-camera model which is with fewer parameters than the original model. Furthermore, a new calibration method is also proposed which can be used to calibrate an omni-camera using only one straight line with no knowledge about its position, direction, or length. Comparing with existing omni-camera models, the proposed one has an great advantage since it can be calibrated reliably from one straight line, so it facilitates a non-technical user to conduct the calibration process without difficulty because it requires no extra calibration target.
10
2.2 Space Line Detection Techniques for Omni-cameras by Equal-width Curve Extractions
An improved detection method is proposed in this study to robustly detect straight lines (i.e., space lines) in the 3D world from the images captured by omni-cameras. The proposed method detects the space lines from the captured omni-images directly without unwarping the captured omni-images, so the processing time may be faster than the methods needing unwarpings. Traditionally, the Hough Transform can be used to detect space lines from omni-images without unwarping it; however, it is showed in this study that the traditional ways all have some problems when detecting the space lines, so the results are imprecise and unrobust. In this study, we first identified three main different conventional approaches to detect space lines, and analyzed the reasons of the imprecision and unrobustness. From the analyze results, it is figured out that an equal-width curve extraction technique can be used to yield a more precise and robust results when detecting space lines. As a result, a technique to extract equal-width curves is proposed using total differential concepts; consequently, an improved Hough transform technique is proposed to detect equal-width curves using the previously-mentioned equal-width extraction method.
From the experimental results, the proposed line detection method can detect space lines more precisely and robustly then the conventional methods. To the processing time, the proposed method only requires a bit longer running time than the conventional ones, but produces a much more precise results than the conventional ones.
2.3 Automatic Adaptation Techniques of Binocular Omni-vision Systems to Any System Setup
From the viewpoint of consumers, one of the important facts when deciding to buy a new system is the convenience of the system setup process, so a system with a convenient setting up process is very important in consumer electronics. In this aspect, a new binocular omni-vision system is proposed in this study, which can be easily deployed by users without any restrictions on the locations or orientations of
11
the cameras, and then the system can automatically adapt system parameters using only the straight lines in the environment. Specifically, the proposed vision system, as shown in Fig. 2.1, consists of two omni-cameras facing the user’s activity area.
Each camera is affixed firmly to the top of a rod, forming an omni-camera stand, with the camera’s optical axis adjusted to be horizontal (i.e., parallel to the ground).
After the two cameras are placed freely by the user with arbitrary locations and orientations, we utilize the straight lines within the surrounding environment as a hint to tell us the orientations of the cameras, and then use these lines to derive the two cameras’ orientations. After deriving the orientations, the system will ask the user to stand at the middle region in front of the two cameras, and derive the distance between the two cameras (i.e., the baseline) with the use of the user’s height. After deriving the orientations and the baseline, a coordinate system can be defined with no ambiguity, so the 3D data can be computed correctly. To further improve the correctness and robustness of the adaptation process, we also take the advantage of the property that the straight lines are mostly parallel or perpendicular to each other, so that the proposed adaptation process can be conducted without finding the line correspondences among the two omni-cameras, and in the meantime improves the correctness and robustness of the adaptation results.
To sum up, we proposed a new 3D vision system using two omni-cameras, which has a capability of automatic adaptation to any system setup for convenient in-field uses. The cameras are allowed to be placed freely in the environment at any location in any orientation, resulting in an arbitrary system setup. Then, by the use of space line features in environments, the proposed vision system can adapt automatically to the arbitrarily-established system configuration by just asking the user to stand still for a little moment in the middle region of the activity area in front of the two cameras. Contrarily, in traditional vision systems, the two cameras may be required to be parallel to each other, and the distance between the cameras may be required to be a fixed length. After this adaptation operation, 3D data can be computed correctly and precisely.
12
Fig. 2.1 The proposed binocular omni-vision system.
2.4 Optimal Design and Placement of Omni-cameras in Binocular Vision Systems for Accurate 3D Data Measurement
The optimal design of a vision system is an important issue for system deployments, and in this study, the optimization problem of designing an binocular omni-vision system is analyzed and solved by three different optimization methods.
In a binocular omni-vision system, which is composed of two catadioptric omni-directional cameras with hyperboloidal-shaped mirrors, an optimal system design includes the optimal shapes of the two hyperboloidal mirrors, the optimal viewing angles of the perspective cameras, the optimal locations of the cameras, and the optimal directions of the camera, and in this study, we focused on finding a such configuration which can yield the most accurate 3D measurements. To solve this optimization problem, the first step is to design a function to estimate the goodness of a system configuration, and then design optimization methods which can minimize the function. In more detail, a criterion function is proposed in this study to estimate the accuracy of the 3D measurements yielded by a system configuration.
That is, the criterion function takes the system parameters as its input arguments, including the camera poses, camera parameters, and the location of the feature point
13
whose 3D data is going to be measured, and produces a value indicates the accuracy of the 3D data yielded by the binocular omni-vision system.
For the criterion function, we use error propagation analysis techniques to estimate the accuracy of the 3D measurement, and the proposed criterion function is with analytical formulas, making it possible to design an analytical and non-iterative optimization method. After defining the criterion function, it is then used in an optimization framework to find the optimal system configurations for different shapes of system setup environments. For regular cases with rectangular cuboid-shaped 3D measurement and camera placement areas, two fast algorithms are proposed to solve the problem, one being bisection-based and relatively slower for deriving the optimal solution; and the other faster using analytic formulas for deriving a sub-optimal solution which is proved to be close to the optimal one in precision. For general cases, an iterative optimization method is proposed along with several speeding-up techniques to accelerate the optimization process. Experimental results of simulations and real application cases show the feasibility of the proposed optimization methods.
2.5 An Omni-vision-based Indoor Parking Lot System with the Capability of Automatic Parking Space Detection
A convenient indoor vision-based parking lot system using wide-angle fisheye-lens or catadioptric cameras is proposed, which is easy to set up by a user with no technical background. Easiness in the system setup comes mainly from the use of a new camera model which can be calibrated using only one space line without knowing its position and direction, as well as from the automatic detections of the parking space boundaries. Comparing with traditional parking lot systems, the traditional ones usually use perspective cameras, rather than use wide-angle cameras, such as fisheye-lens or catadioptric ones, are not commonly adopted yet.
Furthermore, another one problem exists in the traditional systems is the complicated system setup procedure, including camera calibration, environment learning, object modeling, etc., whose implementation usually requires the user to have a lot of technical knowledge. From these viewpoints, an intelligent vision-based system using omni-cameras for parking lot management is proposed, which has the
14
following two merits: 1) the camera system can be set up easily by a common user with no technical background; 2) parking spaces can be detected precisely; and 3) vacant parking spaces can be identified automatically for convenient car parking.
In more detail, the omni-cameras mounted on the ceiling can be easily calibrated using straight lines in the environment using the new camera model and the new calibration method proposed in this study. After camera calibration is conducted, parking-space boundary lines are extracted automatically from input omni-images by a modified Hough transform with a new cell accumulation scheme, which can generates more accurate equal-width curves using the geometric relations of line positions and directions. To further improve the detection results of the parking-space boundary lines, the property that the boundary lines are either parallel or perpendicular to each other is taken into consideration to improve the results.
After the boundary lines are detected, the user may easily add or remove the boundary lines by single clicks on images, and parking spaces can be segmented out by region growing by the use of the boundary lines. Finally, vacant parking spaces can be detected by a background subtraction scheme. A real vision-based parking lot has been established and relevant experiments conducted. Good experimental results show the correctness, feasibility, and robustness of the proposed methods.
15