After the data region is accurately sampled, the intensity values in the data region are still highly polluted by the blurring effect. The blurring effect can be interpreted as a low pass filter which eliminates the significant details of the transmitted pattern, i.e., a loss of the high frequency content. As a result, the data in the high-frequency channels would be severely attenuated, leading to the decoding error in screen-to-camera communications.
In general, the blurring effect is mainly caused by the non-alignment between the pixel blocks in the transmitted code and the pixels in the image sensor. The non-alignment hap-pens when the pixel cell of the image sensor observes the intensity of multiple pixel blocks in the transmitted code. As a pixel cell outputs the total intensity value within its sensing area, the output intensity would be the combination of all the pixel blocks observed. Non-alignment can happen at any distance. For example, at short distance, the non-Non-alignment takes place when the image sensor observes the boundary of two transmitted pixels, which not only reduces the sharpness but also high frequency information of the transmitted im-age. On the other hand, when the distance between the transmitter and the camera is long,
one pixel cell in the image sensor observes multiple transmitter pixels, outputting a sum-mation of the observing pixel values.
In addition to the non-alignment, at long communication distance, the insufficient sharpness of the lens and the defocusing of the camera aggravate the influence of the blurring effect. This is due to that the imperfectness of the lens structure would make the incident light slightly scatter when passing through the lens, resulting in color mixing.
Moreover, even the small movement of human body such as shaking or jittering when taking a picture would result in blurring as well.
To mitigate the blurring effect in the data region, in this work, we treat the blurring effect as another kind of sampling problem. But unlike the resampling method in which the phase shift in the frequency domain is translated into the pixel offset in the spatial domain, we estimate the channel caused by the blurring effect in the frequency domain.
To do so, we take advantage of the pilots inserted in the transmitted code as mentioned before. Then, we can obtain the phase shift difference between the transmitted data and received data. We calculate blurring effect channel by fitting it to a two dimensional plane.
Finally, the received data could be restored by dividing the estimated channel expressed as the following equation:
Chapter 5
Implementation
5.1 Encoder
We use a Matlab program to generate our transmitted code. The algorithm can be divided in four steps:
1. Modulate the transmitted data with PSK.
2. Insert the modulated data and pilots into the upper half of a square matrix, and insert the complex conjugate form of the data in the lower half of the matrix to form a Hermitian matrix.
3. Apply 2D IFFT to the data matrix to obtain the intensity values in spatial domain. Since the result from 2D IFFT ranges from -1 to 1, we need to map the result to 0 to 1 for showing the image.
4. Finder and gradient patterns are added to create a complete transmitted code, as shown in Figure 5.1(a).
To perform nonlinearity calibration, the transmitter obtain the intensity curve calcu-lated by the receiver. According to the intensity curve, the transmitter adjusts the pixel intensity of the transmitted code. As an example, Figure 5.1(b) shows an example of the transmitted code after nonlinearity calibration.
(a) Before nonlinearity calibration. (b) After nonlinearity calibration.
Figure 5.1: An example of nonlinearity calibration.
5.2 Decoder
After we captured the transmitted code with an image, we need to find the finder patterns to locate the data region in the captured image. To enhance the efficiency and accuracy, we apply some image processing technique to first filter out the noise. The steps are detailed as follows:
1. Due to the format of the captured image, which is BMP for the Flea3, we need to trans-late the image from RGB color space to grayscale space. Then, we binarize the grayscale image with the use of Otsu’s method to obtain a proper threshold.
2. The Bayer pattern in the image sensor would cause uneven exposure in the captured image, resulting in a jagged edge of the contour of the finder patterns. To remove the noise, we apply the opening technique in morphological image processing to restore the original contour of the finder pattern.
3. To compute the location of the four finder patterns in the binary image, we search the positions of the finder pattern in which the ratio of the dark and light blocks is 1:1:3:1:1 in both horizontal and vertical direction. Then, the center of the finder patterns could be located.
4. According to the center of four finder patterns, we could locate the position of the data region and calculate the sampling point by 2D interpolation.
After finishing the above steps, the sampling data would be transformed into the fre-quency domain by 2D FFT. Resampling method and channel removal will be applied to adjust the sampling point and restore the erroneous bits. Eventually, the data would be decoded from the transmitted code.
Chapter 6 Evaluation
In this chapter, firstly, we describe the experimental setup. Then, we introduce the key ratio which is used to enable the fair comparison among multiple transmitters for evaluating the system performance. Finally, we analyze the effectiveness of our proposed methods based on the bit error rate (BER) at different communication distances.