|Table of Contents|

Augmented semi-virtual reality cockpit technology(PDF)

《交通运输工程学报》[ISSN:1671-1637/CN:61-1369/U]

Issue:
2012年02期
Page:
62-69
Research Field:
载运工具运用工程
Publishing date:

Info

Title:
Augmented semi-virtual reality cockpit technology
Author(s):
TANG Yong; GU Hong-bin; LI Peng
School of Civil Aviation,Nanjing University of Aeronautics and Astronautics,Nanjing 210016,Jiangsu,China
Keywords:
semi-virtual reality cockpit;  human-computer interaction;  augmented virtuality;  hand image segmentation;  virtual-real matching;  virtual-real fusion
PACS:
V223;TP391.9
DOI:
-
Abstract:
In order to enhance the visual sense of users and the interaction performance of human-cockpit system in the semi-virtual reality cockpit system,an augmented semi-virtual reality cockpit technolgy was presented,which incorporated a realistic view of hand into the virtual environments and made user see his own real hand in real time.The captured images by using camera were real-time segmented by skin color detecting algorithm.The segmented foreground images were rectified by using adaptive weighted stereo matching algorithm and view morphing algorithm.Hand images were seamlessly integrated into virtual environment by using virtual-real matching and virtual-real fusion methods,which can provide user with the visual images in augmented virtual environment.Experimental result shows that augmented semi-virtual reality cockpit technology make users feel true for the feedback of touch and vision.The smallest side length of the square clicked by users is 8 mm in head-mounted virtual environment.In augmented semi-virtual reality cockpit system,the average interaction time decreases by about 50% and 18% compared with traditional data glove method and semi-virtual reality method.The technolgy effectively improves the human-cockpit interaction performance of the system,enhances its practicability,and its developed cost decreases.1 tab,11 figs,22 refs.

References:

[1] GU Hong-bin, WU Dong-su, LIU Hui. Development of a novel low-cost flight simulator for pilot training[J]. World Academy of Science, Engineering and Technology, 2009(60): 685-689.
[2] MCCARTY W D, SHEASBY S, AMBURN P, et al. A virtual cockpit for a distributed interactive simulation[J]. IEEE Computer Graphics and Applications, 1994, 14(1): 49-54.
[3] HOSSEINI M, MALRIC F, GEORGANAS N D. A haptic virtual environment for industrial training[C]∥IEEE. IEEE International Workshop on Haptic Virtual Environments and their Applications. Ottawa: IEEE, 2002: 25-30.
[4] 任 程,戴树岭.基于数据手套的逼真虚拟手的实现[J].系统仿真学报,2008,20(22):6214-6217. REN Cheng, DAI Shu-ling. Implementation of cyber-glove based realistic virtual hand[J]. Journal of System Simulation, 2008, 20(22): 6214-6217.(in Chinese)
[5] O'HAGAN R G, ZELINSKY A, ROUGEAUX S. Visual gesture interfaces for virtual environments[J]. Interacting with Computers, 2002, 14(3): 231-250.
[6] TOSAS M, LI B. Virtual touch screen for mixed reality[C]∥HCI/ECCV. Computer Vision in Human-Computer Inter-action of Lecture Notes in Computer Science. Prague: Springer-Verlag, 2004: 48-59.
[7] FANG Gao-lin, GAO Wen, ZHAO De-bin. Large vocabulary sign language recognition based on fuzzy decision trees[J]. IEEE Transactions on Systems, Man, and Cybernetics, 2004, 34(3): 305-314.
[8] GAO Wen, MA Ji-yong, WU Jiang-qin, et al. Sign language recognition based on HMM/ANN/DP[J]. International Journal on Pattern Recognition and Artificial Intelligence, 2000, 14(5): 587-602.
[9] 郝爱民,何 兵,赵沁平.虚拟现实中的增强虚境技术[J].北京航空航天大学学报,2003,29(10):909-913. HAO Ai-min, HE Bing, ZHAO Qin-ping. Review of augmented virtuality technology in virtual reality[J]. Journal of Beijing University of Aeronautics and Astronautics, 2003, 29(10): 909-913.(in Chinese)
[10] 戴树岭,雷小永,梅继红.虚拟仿真飞机座舱系统[J].系统仿真学报,2002,14(4):488-492. DAI Shu-lin, LEI Xiao-yong, MEI Ji-hong. Virtual cockpit system[J]. Journal of System Simulation, 2002, 14(4): 488-492.(in Chinese)
[11] SALZMANN H, FROEHLICH B. The two-user seating buck: enabling face-to-face discussions of novel car interface concepts[C]∥IEEE. IEEE Virtual Reality 2008. Keno: IEEE, 2008: 75-82.
[12] 周 来,顾宏斌,孙 瑾,等.半虚拟现实座舱中基于最大后验概率框架的手姿态估计[J].航空学报,2011,32(7):1252-1259. ZHOU Lai, GU Hong-bin, SUN Jin, et al. Hand pose estimation based on maximum a posteriori framework in semi-virtual reality cockpit[J]. Acta Aeronautica et Astronautica Sinica, 2011, 32(7): 1252-1259.(in Chinese)
[13] 罗 斌,姚 鹏,翁冬冬,等.基于混合现实的新型轻量级飞行模拟器系统[J].系统仿真学报,2009,21(17):5406-5410. LUO Bin, YAO Peng, WENG Dong-dong, et al. Novel, lightweight flight simulator system based on mixed reality[J]. Journal of System Simulation, 2009, 21(17): 5406-5410.(in Chinese)
[14] ZHANG Zheng-you. Flexible camera calibration by viewing a plane from unknown orientations[C]∥IEEE. Proceedings of the Seventh IEEE International Conference on Computer Vision. Los Alamitos: IEEE, 1999: 666-673.
[15] 徐战武,朱淼良.肤色检测最优空间[J].计算机辅助设计与图形学学报,2006,18(9):1350-1356. XU Zhan-wu, ZHU Miao-liang. Optimum colorspace for skin-tone detection[J]. Journal of Computer-Aided Design and Computer Graphics, 2006, 18(9): 1350-1356.(in Chinese)
[16] 汤 勇,顾宏斌,周 来.交互系统中实时手势分割及指尖检测方法[J].光电工程,2010,37(7):145-150. TANG Yong, GU Hong-bin, ZHOU Lai. Real-time hand segmentation and fingertip detection for interaction[J]. Opto-Electronic Engineering, 2010, 37(7): 145-150.(in Chinese)
[17] EDWARDS E K, ROLLAND J P, KELLER K P. Video see-through design for merging of real and virtual environments[C]∥IEEE. Proceedings of IEEE Virtual Reality Annual Inter-national Symposium. New York: IEEE, 1993: 223-233.
[18] ROLLAND J P, BIOCCA F A, BARLOW T, et al. Quantification of adaptation to virtual-eye location in see-thru head-mounted displays[C]∥IEEE. Proceedings of the Virtual Reality Annual International Symposium. Los Alamitos: IEEE, 1995: 56-66.
[19] TANG Yong, GU Hong-bin. Automatic virtual view gener-ation based on view morphing[J]. Journal of Computational Information Systems, 2011, 7(6): 2051-2057.
[20] YOON K J, KWEON I S. Locally adaptive support-weight approach for visual correspondence search[C]∥IEEE. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington DC: IEEE, 2005: 924-931.
[21] ZITNICK C L, KANADE T. A cooperative algorithm for stereo matching and occlusion detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(7): 675-684.
[22] SCHIEFELE J, ALBERT O, VAN LIER V, et al. Simple force feedback for small virtual environments[C]∥SPIE. Conference of Modeling and Simulating Sensory Response for Real and Virtual Environments. Orlando: SPIE, 1998: 100-110.

Memo

Memo:
-
Last Update: 2012-04-30