CHAPTER 1 INTRODUCTION
1.4 Organization of the Thesis
The rest of the thesis is organized as follows. Chapter 2 describes the proposed
system for decoding MaxiCode. The experimental results and the conclusion are
discussed in Chapter 3 and Chapter 4.
CHAPTER 2
PROPOSED METHOD
In our system, we process the image taken from a camera mobile phone.
Considering the limitation of computing ability of the mobile phone, the size of the
image is set to 640 x 480, and the MaxiCode is located in the central of the image.
The flow diagram of the decoding system is shown in Figure 2.1. The whole
process consists of five major phases: image binarization, finder pattern location,
orientation correction, code shape rectification, and data extraction. In the image
binarization phase, Sobel [8] operator is used as the border point detector, and the
histogram of border points is used to decide the threshold value. In the finder pattern
location phase, a largest connected component detection method is implemented to
locate the MaxiCode’s finder pattern. In the orientation correction phase, the
MaxiCode’s orientation patterns are identified, and the image is rotated to the right
position according to the orientation patterns. In the code shape rectification phase,
the boarder of MaxiCode is detected, and the shape of MaxiCode is rectified by
inverse perspective transformation. In the data extraction phase, a run-length counting
measure is provided to extract the MaxiCode’s data, and then the data is decoded
according to the MaxiCode’s specification.
Fig. 2.1 Flow diagram of the decoding system.
2.1 Symbol Description of MaxiCode
Each MaxiCode (see Fig. 1.1) consists of a central finder pattern surrounded by a
square array of rows of hexagonal modules. The 33 rows in the symbol alternate
between 30 and 29 modules in width. The MaxiCode is surrounded on all four sides
by quiet zone boarders.
2.1.1 Single Module
A single module in MaxiCode is used to encode one bit of data. The module is a
regular hexagonal shape. Figure 2.2 shows an example of a single module.
Fig. 2.2 An example of a single module.
2.1.2 Row of Module
Rows of single modules make up the shape of MaxiCode. In general, there are 30
Data Extraction Code Shape
Rectification Orientation
Correction Finder Pattern
Location Image
Binarization Input
Image
Decoded Message
modules in each odd row and 29 modules in each even row. The start position of the
first module in each even row is half module behind that of each odd row. Figure 2.3
shows an example of odd rows and even rows. There is a white gap between any two
adjacent modules. Figure 2.4 shows an example of adjacent black modules.
Fig. 2.3 An example of odd rows and even rows.
Fig. 2.4 An example of adjacent black modules.
2.1.3 Finder Pattern
The finder pattern is located in the center of MaxiCode. It is made of 3 dark
concentric circles and 3 included white areas. Figure 2.5 shows an example of the
finder pattern relative to the adjacent modules.
odd row
even row
Fig. 2.5 An example of the finder pattern relative to the adjacent modules.
2.1.4 Orientation Pattern
Orientation patterns give the orientation information of MaxiCode. There are 6
orientation patterns in MaxiCode, each consists of 3 modules. These patterns surround
the finder pattern with the position in 1, 3, 5, 7, 9, 11 O’clock. From the aspect of the
finder pattern, the angle between two neighboring patterns is 60º. In each pattern,
there is a module nearest to the outside circle of the finder pattern. The distance
between the outside circle and this nearest module is the length of a single module.
Figure 2.6 shows an example of the orientation patterns relative to the finder pattern.
Fig. 2.6 The orientation patterns relative to the finder pattern, the orientation patterns are marked by W and B.
2.1.5 Quiet Zone
Each MaxiCode is surrounded by four quiet zone boarders. Each boarder consists
of several long white runs, and the number of white runs is the length of a single
module. Figure 2.7 shows an example of the quiet zone.
Fig. 2.7 An example of the quiet zone surrounded by the red dotted lines.
quiet zone
2.2 Image Binarization
The image binarization plays an important role in the decoding system. A good
binary thresholding method can reduce the degradation of poor quality image under
an unstable lighting environment. Before thresholding, we convert the image into gray
level by using
16
where R(i,j) denotes the red value at pixel (i,j), G(i,j) denotes the green value at pixel
(i,j) and B(i,j) denotes the blue value at pixel (i,j). In our system, an image is divided
into several subimages with size 64 x 48, and a local thresholding algorithm is applied
to each subimage. Figure 2.8 shows an example of a subimage. We use Sobel
operation to extract the edge point in a subimage, and the information of these edge
points are used to determine the threshold value.
Fig. 2.8 An example of a subimage.
2.2.1 Edge point extraction
One of the features of barcodes is that most of the barcode’s color is black, and
the barcodes are printed on white papers. This feature makes barcode’s edge
recognizable even under extremely light, dark or other unbalanced lighting conditions.
Weszka et al. [9] proposed a method to extract the edge points of an image. By the
concept proposed by Weszka, we use Sobel operators (see Figure 2.9) to extract edge
points. The Sobel value G of each pixel (i,j) in the subimage is defined as
y
Gx is the magnitude of horizontal gradient and Gy is the magnitude of vertical
gradient.
-1 0 1 -1 -2 -1
-2 0 2 0 0 0
-1 0 1 1 2 1 Fig. 2.9 Two Sobel operators Gx and Gy.
Figure 2.10 shows an example of Sobel result. The numbers of Sobel values are
accumulated from low to high, and we find the value above the 80th percentile as a
percentage threshold. Figure 2.11 shows an example of accumulating diagram. A
point is regarded as an edge point if its Sobel value is higher than the percentage
threshold. Figure 2.12 shows an example of extracted edge points.
(a) (b)
Fig. 2.10 An example of Sobel result. (a) Original subimage, (b) The Sobel result of the subimage.
0 20 40 60 80 100 120
0 0.2 0.4 0.6 0.8 1
Accumulating Percentage
Sobel Value
Fig. 2.11 An example of Sobel value accumulating diagram.
(a) (b)
Fig. 2.12 An example of extracted edge points. (a) Sobel result, (b) Black points represent the edge points.
2.2.2 Minimum mean square error
After the edge points are collected, we establish the histogram of edge points,
and the threshold gray value is determined by minimum mean square error method.
This threshold separates the gray values into two clusters and the mean square error
between the two clusters is the minimum of those among other possible two clusters.
Figure 2.13 shows an example of gray value histogram of edge points. The following
Step 1. Let Min be the minimum gray value, Max be the maximum gray value
and P(i) be the number of edge pixels with gray value i.
Step 2. For each value k with Min ≤ k <Max, use k to divide the histogram of the
edge points into two parts. Calculate the two mean values hk,1 for the left part and hk,2
for the right part, and the mean-square error Ek through the following three formulas:
∑
Usually, in the background area, the gray values will not vary too much, so the
square-error is low and the difference of two means (h1,h2) of both sides separated by
kmin-err will be small. Therefore, if |h1-h2| <c, where c is a pre-defined small value, the
subimage is considered to be a background area. In this thesis, c is defined as 10. The
background area will be filled with white pixels. Figure 2.14 shows an example of
minimum mean square error thresholding.
0 5 10 15 20 25 30
0 15 30 45 60 75 90 105 120135 150 165 180 195 210 225 240 255 Gray Level Value
Number of Pixels
Fig. 2.13 An example of gray value histogram of edge points.
(a) (b)
Fig. 2.14 An example of minimum mean square error thresholding. (a) A MaxiCode image. (b) The binary image of (a).
2.3 Finder Pattern Location
MaxiCode has a central unique finder pattern, which is used to locate the
MaxiCode. The finder pattern is made of 3 dark concentric circles and 3 included
white areas. In order to extract the edge of circles of the finder pattern, the thinning
morphological algorithm [8] is used. Considering the efficiency, only the central part
of the image is processed, and the result of the thinned image is stored as another
Figure 2.15 shows an example of thinning result.
Fig. 2.15 An example of thinning result.
We extract the black connected component in the thinning area, and calculate the
size of each component. Since the finder pattern occupies most part of the area, we
can easily recognize the component of the finder pattern by the size. Figure 2.16
shows a thinned finder pattern example. We pick the component containing most
pixels as the outside circle of the finder pattern. By averaging the coordinates of all
pixels in this component, we can determine the coordinates of the central point of the
finder pattern.
Fig. 2.16 A thinned finder pattern example with outside circle located.
outside circle central point
2.4 Orientation Correction
The reading direction of MaxiCode is not limited because the orientation patterns
give the position information. Figure 2.17 shows the location of orientation patterns
and Figure 2.18 shows some examples of MaxiCode images taken from different
directions. We propose an algorithm to detect the orientation pattern. This algorithm
extracts the length of a single module, and finds the orientation patterns by scanning
around the finder pattern. After the orientation patterns are found, the image is rotated
into the right position.
Fig. 2.17 The location of orientation patterns marked by W and B. W is the white module and B is the black module.
2.4.1 Counting single module length
The orientation patterns are constructed by the basic modules of MaxiCode (See
Fig. 2.17). The length of a single module can be used to detect the position of
orientation modules. In order to calculate the length of a single module, we use a
statistical method to find out the most possible length. We calculate the black
run-lengths in the middle part of the image by horizontal scanning. The length of
black runs on the scan line will be recorded. For efficiency, the scanning horizontal
line’s frequency has a 3 pixels step size. Figure 2.19 shows an example of horizontal
scanning.
(a)
(b)
Fig. 2.19 An example of horizontal scanning. (a) The horizontal scanning lines. (b) The step size of scanning lines.
The histogram of black runs’ length will be analyzed to extract the length of a
single module. Figure 2.20 shows an example of the run length histogram. Generally,
we choose the peak value in the histogram as the result. For avoiding local maximum
3 pixels
value influencing the correct solution, we take 3 values as a group to smooth the
histogram. We compare each group by their sum, and the average value of the group
with the maximum sum will be taken as the length of a single module.
0 50 100 150 200 250 300 350
1 2 3 4 5 6 7 8
Run Length
Number of Runs
Fig. 2.20 An example of run length histogram.
2.4.2 Locating Orientation Patterns
There are six orientation patterns in MaxiCode, and in this thesis, we only use
four of them to correct the orientation. The patterns in positions of 3 O’clock, 5
O’clock, 9 O’clock and 11 O’clock are used. Figure 2.21 shows the corresponding
patterns.
We notice that the patterns in 5 and 11 O’clocks both have two continuous black
modules lying on the same line crossing the center point of the finder pattern, so we
use this feature to find these two orientation patterns. We take the central point of the
finder pattern as the original point, and generate the scanning line from the original
point with different angles. Figure 2.22 shows the concept of scanning lines. We
generate 360 lines with angles from degree 0º to degree 359º. The length of white runs
and black runs in each scanning line are recorded. We remove the first 6 runs of each
line since they are caused by the finder pattern, then we start to analyze the remaining
runs.
Fig. 2.22 The concept of scanning line.
2.4.2.1 Orientation pattern runs analysis
Firstly, we use the length of runs as a condition to choose the possible orientation
pattern runs in each scanning line.All runs appearing in the same scanning line are
grouped into a run set. We prune the run sets by following rules:
Rule 1. The length of the first white run in a run set should be less than or equal
to the length of a single module.
Rule 2. The number of black runs in a run set should be more than or equal to 2.
Secondly, we group the neighboring run sets in the remaining run sets. Note that
two run sets appearing in two scanning lines SL1 and SL2 are called neighbors, if the
difference of the angles of SL1 and SL2 is 1º. Figure 2.23 shows an example of
neighboring run sets. Then we prune the run sets according to following rules:
Rule 3. The number of run sets in a group should be similar to the length of a single module.
Fig. 2.23 An example of a group of neighboring run sets with 6 run sets.
6
Next, we check the symmetry of the remaining run set. The orientation patterns
in 5 and 11 O’clock are symmetrical, that is, the difference of the angles of their
scanning lines should be 180º. We prune the run sets by their symmetry according to
following rules:
Rule 4. If a run set belongs to the orientation patterns, there should exist a
symmetrical run set in the remaining run sets.
After these pruning steps, we may have several pairs of run set groups. Figure
2.24 shows an example of pruning result. Next, we use the feature of the orientation
patterns in 3 and 9 O’clock to prune run set groups. In these two patterns, the module
nearest to the finder pattern is always black, and the angles between 3 to 5 O’clock, 9
to 11 O’clock are both degree 60º. Figure 2.25 shows an example of finding 3 and 9
O’clock patterns. Therefore, we prune the group of run sets by following rules:
Rule 5. For a pair of groups of run sets, locate the first black module in the
degree plus 60º for the first run set of each group. The distance from the finder pattern
to these two black modules should be the same.
Figure 2.26 shows an example of the distance from the finder pattern to a black
module.
(a) (b)
Fig. 2.24 An example of pruning result. (a) The original image. (b) The remaining run set groups pointed out by the arrows.
Fig. 2.25 An example of finding 3 and 9 O’clock patterns. The arrows point out the 3 and 9 O’clock orientation patterns.
By these rules, the two black modules of orientation patterns in 5 and 11
O’clocks are extracted. The final step is to identify the exact positions (5 or 11
O’clock) of the pair of two black modules. Note that the third module of the
orientation pattern is white in 5 O’clock and it is black in 11 O’clock. According to
this phenomenon, we generate a clockwise scanning arc to check the color of the third
module. We use the finder pattern center as the center point and the middle point of
the two black modules as the starting point of the arc. The radius of the arc is the
distance from the central point of the finder pattern to the central point of the two
black modules. Figure 2.27 shows an example of scanning arc. By the color of the
third module, the exact position of the two black modules can be determined.
(a) (b)
Fig. 2.27 A example of the scanning arc. (a) The scanning arc crossing the third module of the 11 O’clock orientation pattern. (b) The scanning arc crossing the third
module of the 5 O’clock orientation pattern.
center point starting point
starting point center point
Finally, we rotate the image of MaxiCode to the right position according to the
found orientation patterns. Figure 2.28 shows an example of rotated MaxiCode image.
(a) (b)
Fig. 2.28 An example of the rotated MaxiCode image. (a) Original image. (b) Rotated image.
2.5 Code Shape Rectification
The MaxiCode image could have a deformed shape since the position between
the barcode and the camera mobile phone is not paralleled. Figure 2.29 shows some
examples of deformation. In our system, slight deformation of MaxiCode image can
be recovered. Inverse perspective transformation is often used to rectify the code
shape. However, the four corner points of MaxiCode should be extracted before using
inverse perspective transformation. Thus, we provide an algorithm to extract the
boarders of the MaxiCode, and take the intersection points of boarders as the corner
Fig. 2.29 Some examples of MaxiCode image deformation.
2.5.1 Boarder Extraction
In the specification of MaxiCode, every MaxiCode is surrounded by a quiet zone
boarder. The color of quiet zone is white, and the width of quiet zone equals to the
length of a single module. In our algorithm, the quiet zone will be roughly extracted.
In the region surrounded by the quiet zone, the most suitable boarder of MaxiCode
will be determined.
2.5.1.1 Quiet Zone Detection
To find the quiet zone, the objective is to find four white runs surrounding the
MaxiCode. Each of the white run should be a long run and the nearest run to the
MaxiCode. We horizontally and vertically scan from the central point of the finder
pattern, and find the first appearance of a group of long white runs. If the long white
runs continually appear with number 2 times of the length of a single module, we
recognize it as the quiet zone. Figure 2.30 shows an example of quiet zone. The quiet
zone can be regarded as the rough boarder of the MaxiCode, and then we extract the
precise MaxiCode boarders in the region surrounded by the quiet zone.
Fig. 2.30 An example of quiet zone marked by red color.
2.5.1.2 Swing to find boarder
In order to use inverse perspective transformation, we need to find the four
corner points of MaxiCode precisely. We find the nearest point to each rough boarder,
and take this point as the pivot. Then we generate scanning lines which swing with
this pivot, and the swinging angle ranges from -3.0º to 3.0º degrees, adding 0.2º
degrees per move. Figure 2.31 shows an example of swinging scanning lines. We take
the scanning line touching the most modules as the precise boarder. Figure 2.32 shows
an example of MaxiCode with extracted boarder. Finally, we take the four intersection a group of white
runs
Fig. 2.31 An example of swinging scanning lines.
Fig. 2.32 An example of MaxiCode with extracted boarder.
2.5.2 Inverse Perspective Transformation
The following equations calculate coefficients of perspective transformation [7]
which maps vertexes (xi,yi) to vertexes (ui,vi) (i=1,2,3,4):
Coefficients are calculated by solving linear system:
pivot
chosen scanning line
⎥⎥
Let f(x,y) represent the pixel value in position (x,y) in the rectified image, and
g(u,v) represent the pixel value in position (u,v) in the original image. The rectified image is mapped from the original one by the following equation:
⎟⎟⎠
The resolution of the rectified image is defined as 210 x 200. Figure 2.33 shows
an example of rectified MaxiCode image.
(a) (b)
Fig. 2.33 An example of rectified MaxiCode image. (a) Original image. (b)
2.6 Data Extraction
To decode the information embedded in the MaxiCode, we need to extract the
data from modules. There are 33 rows of modules in a MaxiCode, so we separate the
image into 33 parts in rows, and recognize the modules in each row by run length
counting. After all modules being recognized, the data are decoded by the
specification of MaxiCode. Finally, the result of decoding will be shown on the screen
of the mobile phone.
2.6.1 Module run length counting
In a MaxiCode, there are 30 modules in each odd row and 29 modules in each
even row. Black modules represent 1 and white modules represent 0. We calculate the
white runs and black runs in each row, and analyze how many modules in each row.
The finder pattern is removed at the beginning of this stage to avoid influencing the
counting of run lengths.
In each row, we choose the scanning line which contains the most number of
In each row, we choose the scanning line which contains the most number of