3.2 Polygon Extraction
Vanishing points prediction
Local image features have been shown to be a powerful tool for further inference of the context in an image, such as vanishing points and vanishing lines. Under perspective projection, parallel lines in 3D world appear to converge at a point in the image called vanishing point (VP). Lines that meet at the vanishing points is considered to be vanishing
14
Figure 3.2 A example of vanishing points and vanishing lines
lines (VL). We assume that there are several horizontal vanishing points in the scene. Since we assume that the vertical structure lines of the architecture are perpendicular to the ground plane, only one vertical vanishing point exists in the scene. The horizontal vanishing lines converge at horizontal vanishing points and the vertical vanishing lines intersect at the vertical vanishing point. The example of vanishing points and vanishing direction is shown in Figure 3.2.
Since the main structural feature lines of the building are likely in correspondence with the vanishing lines, we can utilize this character to find vanishing points in the image.
(See Figure 3.2.)To estimate the possible location where the vanishing points are, a voting-based prediction method is adopted as well. First, we divide the feature lines detected by Hough transform into horizontal line group and vertical line group according to their orientation. Then, we utilize these two line groups of structure features to estimate the horizontal vanishing points and the vertical vanishing point.
15
We formulate a probability model for predicting the potential location vp as a vanishing point in the image I. The objective is to learn this model from a set of possible vanishing lines from the Hough transform. The feature lines extracted by the Hough transform are used to subsequently cast probabilistic votes for possible vanishing point locations, where the hypothesis score is obtained as a sum over all votes.We define a score function V of the voting procedure as a probability density over the vanishing points location vp = (x, y) in the image I : 1. Extract main structural feature lines of architecture by Hough
transformation.
2. Structural feature lines cast probabilistic votes for possible vanishing point locations by sum over score function of each lines.
3. Pixel x on feature lines votes using a single asymmetric Gaussian kernel to propagate weight effect around x .
4. Apply local minimum suppression on the predicted probability model in the voting space.
5. Search for a strong vanishing point candidate with local maximum score in a region.
6. Select the N vanishing point candidates with top score in the model,
N is user-specified
16
We compute the voting score of every point that feature line passes through :
| ∑ | (3-2)
x is the point on the feature line f. We propagate the voting effect around x with a single asymmetric Gaussian kernel.
By analyzing this voting space, we can identify the peaks to find possible vanishing point candidates. The number of vanishing point candidates can be manual assign.
Non-maximum suppression is used to find a strong vanishing point candidate with local maximum score in a region. The vanishing point prediction method is briefly described in Table I.
Our vanishing point prediction method is focus on single images. There is no 3D information about the image required, such as the camera position and camera parameters.
It is not an iterative approach, thus the computation complexity is low. We employ an efficient voting procedure to find the stable vanishing points. Those vanishing point candidate found in a single image is quite stable and accurate. We can use the result to find where the main vanishing points are located and utilize it as a powerful cue in subsequent process.
It takes very little user intervention for people to correctly infer the locations of vanishing points in a image. Thus, we can also optionally remove unwanted vanishing points from our vanishing point candidates to improve the result.
Line Segment Detection
17
With the main feature lines of the building and predicted vanishing points, we only have preliminary information about the architecture in the image. We still need more cues to infer the structure about the buildings. A line segment detection method [GJMR10] is employed to extract detail line segments that describe the structure of the building.
Figure 3.3 The orientation label of line segments in the scene and the Vertical Corner Line Hypothesis
We group these line segments into three groups according to their relation with the vanishing points. There are line segments vanishing at vertical vanishing point, at main vanishing points and other case. Different labels are attached to each line segment , indicating different group that it belongs. These groups of line segments are utilized to infer the Vertical Corner Line Hypothesis (VCLH) of the building structure. A example of orientation label of
line segments in the scene and the Vertical Corner Line Hypothesis is shown in Figure 3.3.
Vertical Corner Line Hypothesis (VCLH) is a concept that inspired by the paper
“Estimating Camera Pose from a Single Urban Ground-View Omnidirectional Image and a 2D Building Outline Map” [CCTPC10].A Vertical Corner Line Hypothesis is a line in image
18
Group(1) Collinearity with 1 or more vertical aligned line segments and group(2) Collinearity with end points of some line segments sharing a common horizontal vanishing point.For group(1), we search for sets of line segments that have vertical orientation label and check their collinearity with each other in each line segment set. If a set of vertical line segments is over a number of threshold and collinear, we find the best-fitting line for all the endpoints in the set by least square solution. This best-fitting line is a possible VCLH in the image. For group(2), we utilize the line segments which are labeled with different horizontal orientation labels to find other possible VCLHs in the image. By checking the collinearity of the endpoints in different labeled line segments, we can find several sets of endpoint are collinear.
Those collinear endpoints are used to find a best-fitting line through them and obtain possible VCLH.
Preliminary Polygon Extraction
After the vanishing point candidates are obtained, we apply those vanishing points in preliminary polygon extraction. A preliminary polygon is a region that contains approximate façades of the building, and its orientation satisfies the vanishing direction with vanishing points. The preliminary polygon is just an intermediate result of extraction process, and it still requires a refinement process to complete the final façade extraction result.
19
The vanishing point candidates are used to judge whether a feature line is a valid vanishing line according to its orientation, i.e. this line passes through the vanishing point candidates and their nearby regions. We choose the upper most and the lower most valid feature lines to be the boundary lines of the preliminary polygon. A reference vertical feature line is also selected to be the boundary line according to the position of the main vanishing points in the image.
20