Visible Light Communications Using Mobile
Phone Camera
Volume 7, Number 5, October 2015
Chi-Wai Chow, Senior Member, IEEE
Chung-Yen Chen
Shih-Hao Chen
DOI: 10.1109/JPHOT.2015.2476757
1943-0655
Ó 2015 IEEE
Chi-Wai Chow,Senior Member, IEEE, Chung-Yen Chen, and Shih-Hao Chen
Department of Photonics and Institute of Electro-Optical Engineering, National Chiao Tung University, Hsinchu 30010, Taiwan
DOI: 10.1109/JPHOT.2015.2476757
1943-0655Ó 2015 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Manuscript received August 20, 2015; revised August 31, 2015; accepted September 1, 2015. Date of current version September 11, 2015. This work was supported by the Ministry of Science and Tech-nology of Taiwan under MOST-104-2628-E-009-011-MY3 and MOST-103-2221-E-009-030-MY3; by the Aim for the Top University Plan, Taiwan; and by the Ministry of Education of Taiwan. Corresponding author: C.-W. Chow (e-mail: [email protected]).
Abstract:Visible light communication (VLC) is attractive as a supplementary method for future fifth-generation (5G) wireless communication owing to the shortage of radio-frequency bandwidth in conventional wireless communications. The VLC systems reported in the litera-ture are mainly based on PIN receivers (Rxs). It is highly desirable if these VLC signals can be detected by using built-in complementary metal–oxide–semiconductor (CMOS) cameras as Rxs to provide flexible and low-cost wireless communications. However, using the CMOS camera is challenging. In this paper, we propose and demonstrate a VLC link using a CMOS mobile phone camera as Rx. By using the rolling shutter effect of the CMOS sensor, the VLC data rate can be significantly enhanced. We first use a second-order polynomial fitting to miti-gate the“blooming effect” (saturation of pixels) of the CMOS sensor. In order to extend the VLC transmission distance and mitigate the influence of the background noise, we also pro-pose and demonstrate using histogram equalization and Sobel filter to enhance VLC signal performance. Finally, a third-order polynomial fitting is used to define the threshold. The exper-imental results show that significant improvement of bit error rate (BER) for about an order of magnitude under different illuminances can be achieved.
Index Terms: Free-space communication, optical communications, light-emitting diode (LED).
1. Introduction
Due to the continuous decrease in price and the improvement in lighting efficiency, light-emitting diodes (LEDs) are gradually replacing traditional light sources. Recently, visible light communi-cation (VLC) [1]–[4] has attracted attention as a supplement for future fifth-generation (5G) wire-less communications, owing to the shortage of radio-frequency (RF) bandwidth in conventional wireless communications [5]. Many research efforts have been made to increase the modulation response of the transmitter (Tx) (i.e., LED lamp) [6] and receiver (Rx) [7], [8], as well as mitigat-ing the background optical noises [9], [10]. However, these LED VLC transmission systems are mainly based on PIN photodiode (PD) Rxs. Over the last decade, mobile-phones with the built-in Complementary Metal-Oxide-Semiconductor (CMOS) cameras have become common. Peo-ple can take photos and videos whenever and wherever they want. Hence, it is highly desirable if these VLC signals can be received by using the built-in CMOS cameras to provide flexible
and low-cost wireless communications. However, using the CMOS camera as the VLC Rx is challenging since the typical frame rate of the CMOS camera is low, which is about 30 Hz. Ref. [11] describes that the rolling shutter effect of CMOS camera can in principle increase the VLC data rate higher than the frame rate. In [12], a tailor-made image sensor with specific pixels for imaging and PIN PD for VLC, respectively, is described. However, this sensor needs relatively complicated fabrication process and may not be available as an embedded device in future mobile phones.
In this work, we propose and demonstrate a VLC link using CMOS mobile-phone camera as Rx. By using the rolling shutter effect of the CMOS sensor, the VLC data rate can be signifi-cantly enhanced. However, using the rolling shutter effect is challenging. Here, we first use a second order polynomial fitting to mitigate the “blooming effect” (saturation of pixels) of the CMOS sensor, then a column matrix of grayscale values can be obtained. In order to extend the VLC transmission distance and mitigating the influence of the background noise, up to our knowledge, we first propose and demonstrate using histogram equalization [13] and Sobel filter [14] to enhance VLC data signal performance. The histogram equalization is used to enhance the extinction ratio of the grayscale data, and the Sobel filter is used to for the edge detection of the data. The experimental results show that at illuminance of700 lux, the bit-error-rate (BER) improves from 3:25 102 to 6:05 104 when the proposed scheme is applied. For other
illu-minances, the proposed scheme also shows BER improvement.
2. Principle of Using CMOS Sensor as VLC Rx
Most mobile phones have built-in CMOS cameras nowadays; hence, the deployment cost can be much lower if VLC is implemented using the mobile-phone CMOS camera instead of using a plug-in module with PIN PD. However, the CMOS camera has several characteristics which make the implementation challenging. One feature is the rolling shutter effect. It is a method of image acquisition in which each frame is not captured at a single point of time but by scanning or activating horizontally a row of pixels. The schematic diagrams of the global (image captured at a time) and rolling shutters are shown in Fig. 1(a) and (b), respectively. If the LED Tx switches on and off at the frequencies higher than the frame rate of the CMOS camera, bright and dark fringes can be captured in a single image. When the LED Tx is on, bright pixel is stored at the activated row of pixel. When the LED Tx is off, dark fringe is stored at the activated row of pixel. Hence, by computing the bright and dark fringes in an image, the data logic of the VLC wireless communication can be obtained with data rate much higher than the frame rate. After the process is completed, the rows of pixels captured at different time are merged together to form a single image; this time period is called the “read-out time gap,” as shown in Fig. 1. During this period of time, the CMOS sensor is“blind” and cannot detect a signal.
3. Experiment, Results, and Discussion
A proof-of-concept VLC experiment using mobile-phone CMOS camera as Rx is shown in Fig. 2. The transmitting data is programmed using MATLAB in a computer. It is transferred to a LED (Cree XLamp XR-E) via an arbitrary waveform generator (AWG, Tektronix, AFG 3252C),
which acts as a digital-to-analog converter (DAC). The sampling rate and bandwidth of the AWG are 2 GSamples/s and 240 MHz respectively. The LED is phosphor-based emitting white color, with direct modulation bandwidth of1.2 MHz. The modulated white light is then received by a mobile-phone camera (Samsung GT-S7500). The embedded CMOS sensor had a resolution and frame rate of 480 640 and 28 frames per second, respectively. The videos are transferred to a computer for synchronization, demodulation and BER analysis. A 2-min video is captured at each scenario for performance analysis.
The transmitting signal is packet-based including a 4-bit header, 32-bit payload data, and a 1-bit trailer. The header consists of 4-bit Manchester coded signal, and this is enough for our clock recovery at the Rx. The payload data is in on-off keying (OOK) format and the trailer is set to logic 0 in order to distinguish the succeeding packet. In the Manchester coded header, each bit consists of high and low levels; hence, much narrower and dark fringes can be easily located in an image frame when compared with the OOK payload. Besides, the clock information can be easily retrieved from the Manchester coded header. As mentioned in last section, during the read-out time gap between each frame, the CMOS sensor is“blind.” We measure the time gap in our mobile-phone, and it is quite large (14.29 ms, about 40% of the frame duration). Hence, each packet is transmitted three times successively to make sure each image contains enough header information for synchronization. The net data rate is 0.896 kb/s.
In the synchronization and demodulation processes, the video file is converted to MPG for pro-cessing in MATLAB. Each image is converted to grayscale format, in which 0 means completely dark pixel while 255 means completely bright pixel. Then, a vertical column pixel will be selected for the data demodulation. Fig. 3(a) shows an arbitrary image after the grayscale conversion. We can observe the blooming effect of the CMOS sensor at the center of the image, which is due to the overflow of charge from the saturated pixels into the neighboring pixels. If the vertical column pixel is selected at the center, such as the yellow line in Fig. 3(a), many originally dark fringes will be received as bright fringes due to blooming. Hence, the header information cannot be re-trieved, and higher error rate will result. Because of this, we propose using a second-order
Fig. 3. (a) One image showing the bright and dark fringes and the blooming effect at the center. Grayscale values (blue line) and the second order polynomial fitting curve (red line) for (b) bright and (c) dark fringes with blooming.
polynomial fitting for each row pixel. This means there are 480 polynomial fitting curves for each image. Fig. 3(b) and (c) show the grayscale values (blue line) and the second-order poly-nomial fitting curve (red line) for bright and dark fringes with blooming, respectively. Assume each element in a row pixel isðxi; yiÞ, where xi is the ith pixel, and yi is the grayscale value that
pixel, i ¼ 1; 2; . . . 640. We also assume that f ðxiÞ is the second order polynomial fitting curve. By
settingjyi f ðxiÞj G 5, we can get several yi satisfying the above condition, which occur at the
in-terception regions between the blue and red curves in Fig. 3(b) and (c). Then, these values are arranged in ascending order based on their grayscale values, and 20% value is selected. When processing a dark fringe, the selected grayscale value will be approaching to zero; when pro-cessing a bright fringe, as the 20% value is selected, the blooming can be avoided. It is worth to method that this blooming mitigation scheme is independent of the VLC transmission distance and the LED appearance location on the image.
After the blooming mitigation, a column matrix of grayscale values is selected as shown in Fig. 4(a). However, we can observe that the extinction ratio between the bright and dark values is not high; in particular, the extinction ratio of header is very poor. In order to extend the VLC transmission distance and mitigating the influence of the background noise, we also propose and demonstrate using histogram equalization and Sobel filter to enhance VLC signal performance.
We first use the histogram equalization to enhance the extinction ratio of the bright and dark fringes. This scheme is to transform an image with pixels occupying the entire range of gray-scale levels with high uniformity. For the graygray-scale image having nk pixels with grayscale of rk,
the probability density function (PDF) can be described by PrðrkÞ ¼
nk
n ; 0 rk 1; k ¼ 0; 1; 2; . . . ; 255 (1) where k is the number of grayscale value, and n is the total number of pixels in the image. The histogram equalized image after the transform can be represented by
TðrkÞ ¼ Xk j¼0 nj n ¼ Xk j¼0 PrðrjÞ: (2)
As shown in Fig. 4(b), after the histogram equalization, the grayscale values between 0 to 35 are transformed to between 80 to 255. However, after the histogram equalization, the extinction ratio of the periodic header is still low. Then, we apply Sobel filter. The Sobel filter algorithm is used to create an image with emphasized edges. It includes two 3 3 kernels or weighting functions hx and hy, which are convolved with the original image to calculate the derivates. The
hx and hy are for the horizontal and vertical changes respectively, as described by
hx ¼ 1 0 1 2 0 2 1 0 1 2 4 3 5; hy ¼ 1 2 1 0 0 0 1 2 1 2 4 3 5: (3)
Fig. 4. Column matrix grayscale values of (a) original signal, after (b) histogram equalization, and after (c) Sobel filter.
Assuming that the input image is f , after the convolution, as illustrated by
Gx ¼ f hx; Gy ¼ f hy (4)
the image after the Sobel filter will be calculated based on G¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi G2 xþ Gy2 q : (5)
As our input image is 1-D grayscale data, the 1 480 original data cannot convolve with the 3 3 Sobel kernels. In this case, the MATLAB program will patch zeros to form a 3 480 matrix; hence after convolve with the Sobel kernels, a 1 480 output matrix can be obtained, as shown in Fig. 4(c). This means the Gx is obtained. As the original data is one-dimensional,
Gy ¼ 0; hence, G ¼ jGxj. Then, the histogram equalized data adds the Sobel filtered data to
pro-duce the final data as shown in Fig. 5(b). We can observe that the grayscale contrast increases from 35 [Fig. 5(a)) to 175 (Fig. 5(b)] in the new grayscale data. Besides, the extinction ratio of the periodic header is significantly enhanced, with nearly the same extinction ratio as the payload data.
Finally, a proper threshold level should be selected for Fig. 5(b) in order to recover the logic 1 and 0. The third order polynomial fitting is used to define a threshold. Fig. 6(a) and (b) show original grayscale values and the grayscale values after the histogram equalization and Sobel filter, respectively. Third order polynomial fitting are applied to both cases as illustrated using the red curves in Fig. 6(a) and (b). Due to the enhancement of the header and payload data af-ter the proposed schemes, the third order polynomial fitting curve can be properly located to provide a good threshold for logic selection.
Fig. 7 shows the measured BER without and with using the proposed histogram equalization and Sobel filter. Due to the rolling shutter effect, the data rate of the VLC using mobile-phone
Fig. 6. Column matrix of grayscale values with third-order polynomial fitting for (a) original signal after (b) histogram equalization and Sobel filter.
Fig. 5. Column matrix of grayscale values of (a) original signal after (b) histogram equalization and Sobel filter.
CMOS camera can be significantly increased. However, due to the read-out time gap between each frame (40% of the frame duration), each packet is transmitted three times successively to make sure each image contains enough header information for synchronization. By removing the header and trailer bits, the net data rate is 0.896 kb/s. The data rate can be enhanced by using a mobile phone with a better image senor resolution (e.g. 1080 rows); hence, the effective number of bits represented in each image frame can be increased. Besides, it can also be en-hanced by using an image sensor with less read-out time; hence less duplicated packets are needed. From Fig. 7, we can observe that at the illuminance of700 lux, which is corresponding to 25 cm free-space transmission when only a single white-light LED is used, the BER im-proves from 3:25 102to 6:05 104when the proposed scheme is applied. The experimental
results also show that significant improvement of BER for about an order of magnitude under dif-ferent illuminances can be achieved.
4. Conclusion
We first used a second order polynomial fitting to mitigate the “blooming effect” of the CMOS sensor. To extend the VLC transmission distance and mitigating the influence of the background noise, we also proposed and demonstrated using histogram equalization and Sobel filter to en-hance VLC signal performance. The grayscale contrast increased from 35 to 175. The net data rate was 0.896 kb/s. We believed that the data rate in this proof-of-concept demonstration may be enough for transmitting authorization information wirelessly or sending position information [15] in indoor navigation system. The experimental results showed that at the illuminance of 700 lux, the BER improved from 3:25 102 to 6:05 104 when the proposed scheme was
applied. For other illuminances, the proposed scheme also showed BER improvement.
References
[1] C. W. Chow, C. H. Yeh, Y. Liu, and Y. F. Liu,“Digital signal processing for light emitting diode based visible light communication,” IEEE Photon. Soc. Newslett., vol. 26, no. 5, pp. 9–13, Oct. 2012.
[2] H. H. Lu et al., “A multiple-input-multiple-output visible light communication system based on VCSELs and spatial light modulators,” Opt. Exp., vol. 22, no. 3, pp. 3468–3474, Feb. 2014.
[3] Z. Wang, C. Yu, W. D. Zhong, J. Chen, and W. Chen,“Performance of a novel LED lamp arrangement to reduce SNR fluctuation for multi-user visible light communication systems,” Opt. Exp., vol. 20, no. 4, pp. 4564–4573, Feb. 2012. [4] B. Janjua et al., “Going beyond 4 Gbps data rate by employing RGB laser diodes for visible light communication,”
Opt. Exp., vol. 23, no. 14, pp. 18746–18753, Jul. 2015.
[5] S. Wu, H. Wang, and C. H. Youn,“Visible light communications for 5G wireless networking systems: From fixed to mobile communications,” IEEE Netw., vol. 28, no. 6, pp. 41–45, Nov./Dec. 2014.
[6] H. Le-Minh et al., “High-speed visible light communications using multiple-resonant equalization,” IEEE Photon. Technol. Lett., vol. 20, no. 14, pp. 1243–1245, 2008.
[7] J. Y. Sung, C. W. Chow, and C. H. Yeh,“Is blue optical filter necessary in high speed phosphor-based white light LED visible light communications?” Opt. Exp., vol. 22, no. 17, pp. 20646–20651, Aug. 2014.
[8] C. H. Yeh et al.,“Adaptive 84.44–190 Mbit/s phosphor-LED wireless communication utilizing no blue filter at practi-cal transmission distance,” Opt. Exp., vol. 22, no. 8, pp. 9783–9788, Apr. 2014.
[9] C. W. Chow, C. H. Yeh, Y. F. Liu, and P. Y. Huang,“Background optical noises circumvention in LED optical wire-less systems using OFDM,” IEEE Photon. J., vol. 5, no. 2, Apr. 2013, Art. ID. 7900709.
Comput. Graph. Image Process., vol. 8, no. 1, pp. 127–135, Aug. 1978.
[15] Z. Yang, Z. Wang, J. Zhang, C. Huang, and Q. Zhang,“Wearables can afford: Light-weight indoor positioning with visible light,” in Proc. Annu. Int. Conf. Mobile Syst., Appl., Serv., 2015, pp. 317–330.
