Future Work

Following, we hope to further introduce the bare hand 3D multi-touch system.

Considering the relatively low sensitivity of the optical sensor in existence, lighting systems are proposed which are endurable for ambient light effect. However, it is still inconvenient for hand holding equipment in some 3D interaction applications.

Therefore, instead of illuminating the incident light from a light pen or a light glove, the system emits IR light from the back light and thus can be reflected by interaction objects such as fingers.

Based on the embedded optical sensor structure, an additional shielding layer is constructed, as shown in Fig. 92. Through properly design the pattern of shielding layer and the relative position to the optical sensors, the directional optical sensors which mean the optical sensors are able to capture light from different incident angles can be achieved. There are four sensors in a sensing group with a xy-sensor and three z-sensors. The xy-sensor is able to determine the 2D coordinate (x, y) of the objects.

With no blocking pattern above the xy-sensor, the sensor can capture light from all direction. Hence, 2D coordinate (x, y) can be determined by the xy-sensor with maximum intensity. On the other hand, z-sensors can only detect light from specific directions. By knowing the relation between the incident angle and the depth value (z), the depth information can therefore be acquired.

85

Fig. 92. Structure for the bare-hand 3D multi-touch system.

Fig. 93. Working principle of directional sensor by shielding layer.

The working principle of shielding layer is illustrated in Fig. 93. By calculating the 2D coordinate from xy-sensor, z-sensors in the same sensing-group are activated to obtain the depth value. For instance, if an object is positioned around 1 cm above the panel, the maximum reflectance of light should be 45 degree, which is according to the structure design. Hence, the depth value can be calculated by the weighting of z-sensors of the sensing-group by Eq. (10).

Z =𝐼₁× 1 + 𝐼₂× 2 + 𝐼₃× 3 𝐼₁+ 𝐼₂+ 𝐼₃

wbere 𝐼_𝑛 = 𝐼𝑛𝑡𝑒𝑛𝑠𝑖𝑡𝑦 𝑎𝑡 𝑧 − 𝑠𝑒𝑛𝑠𝑜𝑟 𝑓𝑜𝑟 𝑛 𝑐𝑚

Eq. (10)

86

Additionally, multiple objects or fingers can be detected simultaneously. Due to the separation of four sensors in a group, reflected light from different objects can be captured by the accordant sensing-group. Hence a bare-hand 3D multi-touch interaction can be achieved.

87

Reference

[1] C.L. Sholes, C. Glidden, and S.W. Soule, “Improvement in Type-writing Machines,” US 79868, 1868.

[2] N.R. Ball, J.N. Vardalas, Ferranti-Packard: Pioneers in Canadian Electrical Manufacturing. Mcgill Queens Univ Pr, 1994.

[8] J. Schöning, P. Brandl, F. Daiber, F. Echtler,O. Hilliges, J. Hook, “Multi-Touch Surfaces A Technical Guide,” Interactive Surfaces, 2008.

[9] J. Y. Han, “Low-cost multi-touch sensing throughfrustrated total internal reflection,” In UIST ’05: Proceedings of the 18th annual ACM symposium on User interface software and technology, pp. 115–118, 2005.

[10] J. Y. Han, “Multi-touch interaction wall,” SIGGRAPH’06: ACM SIGGRAPH 2006 Emerging technologies, pp. 25, 2006.

[11] E. Costanza and J. Robinson, “A region adjacency tree approach to the detection and design of fiducials,” Vision, Video and Graphics (VVG), pp. 63–70, 2003.

[12] NUI Group Authors, Multi-Touch Technologies, 1^st ed. ch1-6, 2009

[13] G. Walker, “Camera based optical touch technology,” Information Display, pp.

30-34, 2011.

[14] A. Abileah, W.d. Boer, T. Larsson, T. Baker, S. Robinson, R. Siegel, N.

Fickenscher, “Integrated Optical Touch Panel in a 14.1” AMLCD”, Planar Systems Inc.

[15] S. Hodges, S. Izadi, A. Butler, A. Rrustemi, and B. Buxton, “ThinSight: versatile multi-touch sensing for thin form-factor displays,” Proceedings of the 20th annual ACM symposium on User interface software and technology (UIST’ 07), pp. 259-268, 2007.

[16] N. Marquardt, R. Jota, S. Greenberg, and J.A. Jorge, “The continuous interaction

88

space: interaction techniques unifying touch and gesture on and above a digital surface,” In Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction, pp. 461-476, 2011.

[17] N. Holliman, 3D Display Systems. Department of Computer Science, University of Durham, 2005.

[18] D. Valkov, F. Steinicke, G. Bruder, and K. Hinrichs, “2D touching of 3D stereoscopic objects,” In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11), pp. 1353-1362, 2011.

[19] R. Ott, M, Gutiérrez, D. Thalmann, and F. Vexo, “Advanced virtual reality technologies for surveillance and security Applications,” In Proceedings of the 2006 ACM international conference on Virtual reality continuum and its applications (VRCIA '06), pp. 163-170, 2006.

[20] K.N. Tarchanidis, J.N. Lygouras, "Data glove with a force sensor," In IEEE Transactions on Instrumentation and Measurement, vol.52, no.3, pp. 984- 989, 2003.

[21] R. Ott, M, Gutiérrez, D. Thalmann, and F. Vexo, “Vr haptic interfaces for teleoperation: an evaluation study,” In Proceedings of Intelligent Vehicles Symposium, 2005. IEEE, pp. 789- 794, 2005.

[22] T. Schou and H.J. Gardner, “A Wii remote, a game engine, five sensor bars and a virtual reality theatre,” In Proceedings of the 19th Australasian conference on Computer-Human Interaction: Entertaining User Interfaces (OZCHI '07), ACM, pp.

231-234, 2007.

[23] H.J. Luinge, P.H. Veltink, "Inclination measurement of human movement using a 3-D accelerometer with autocalibration," Neural Systems and Rehabilitation Engineering, IEEE Transactions, vol.12, no.1, pp.112-121, 2004.

[24] Microsoft Research, Resources:

http://www.engadget.com/2011/02/25/microsoft-research-shows-off-next-generation-gesture-interfaces/

[25] R. Raskar, H. Nii, B. deDecker, “Prakash: lighting aware motion capture using photosensing markers and multiplexed illuminators,” ACM SIGGRAPH 2007, ACM, New York, NY, USA, article 36.

[26] W.d. Boer, A. Abileah, P. Green, T. Larsson, S. Robinson, and T. Nguyen, “56.3:

Active matrix LCD with integrated optical touch screen,” SID Symposium Digest of Technical Papers, vol. 34, pp. 1494- 1497, 2003.

[27] K. Yi, C. Choi, S. Suh, B. Yoo, J.J. Han, D. Park, and C. Kim, "45.2:

Distinguished paper: Novel LCD display with a sensible backlight," SID Symposium Digest of Technical Papers, vol. 41, pp. 673-676, 2010.

[28] C. Brown, D. Montgomery, J.L. Castagner, H. Kato, and Y. Kanbayashi, "31.3: A system LCD with integrated 3-dimensional input device," SID Symposium Digest of

89

Technical Papers, vol. 41, pp. 453-456, 2010.

[29] H.U. Tung, “3D multi-interactive system,” Optics and Photonics Taiwan, 2010.

[30] C.C. Chao, Guo-Zhen Wang, Yi-Pai Huang, “Multi-Mark Based 3D Interactive System for Multi-user,” TDC 2012.

[31] Guo-Zhen Wang, Ming-Ching Ma, Shang-Yu Tung, Yi-Pai Huang, Hung-Wei Tseng, Jui-Chi Lo, Chung-Hong Kuo, “A Virtual Touched 3D Interactive Display with Embedded Optical Sensor Array for 5-axis Detection” ,SID, pp.737-740, 2011.

[32] L. Lipton, M. Ackerman, “Liquid crystal shutter system for stereoscopic and other applications,” US Patent 4,967,268, 1990.

[33] H. Kim, "Interaction with hand gesture for a back-projection wall,” In Proceedings of Computer Graphics International, pp.395-402, 2004.

[34] M. Fiala, “ARTag, a fiducial marker system using digital techniques”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2005, vol.2, pp. 590- 596, 2005.

[35] T. Matsumoto, L.O. Chua, T. Yokohama, "Image thinning with a cellular neural network," Circuits and Systems, IEEE Transactions on , vol.37, no.5, pp.638-640, 1990.

[36] R.C. Gonzalez, R.E. Woods, Digital Image Processing, 3rd ed. New Jersey:

Prentice Hall, ch9, 2008.

[37] K. Suzuki, I. Horiba, N. Sugie, “Linear-time connected-component labeling based on sequential local operations,” Computer Vision and Image Understanding, Vol.89, Issue 1, pp.1-23, 2003.

[38] L. Shapiro, G. Stockman. Computer Vision. Prentice Hall, 2002, pp. 69–73.

[39] R.C. Gonzalez, R.E. Woods, Digital Image Processing, 3rd ed. New Jersey:

Prentice Hall, ch10, 2008.