基于深度视觉的动态手势交互技术研究
哈尔滨工业大学工程硕士学位论文
Abstract
AR technology is generated in the development of virtual reality technology. The computer draws the virtual image by rendering. The AR technology accurately integrates the virtual image into the real world captured by the camera. Finally, through the video display device, the fused scene is presented to complete the perfect combination of the real world and the virtual world. Nowadays, the interactive system of modern industry has moved away from the virtual environment. Today, as the growing virtual reality and augmented reality technologies, the natural interaction with the virtual environment becomes more and more important. However, technologies interacting with the virtual environment still have some shortcomings. For instance, external objects such as gloves need to be interacted with in a virtual environment. Hardware devices have problems of occlusion and identification when identifying gestures. Based on the above problems, this paper presents a solution, which is the content of this article: based on the depth of visual three-dimensional dynamic hand gestures interaction technology. Using Intel RealSense depth camera combined with depth information and bone movement features to construct the virtual hand model and set up an interactive system to achieve accurate interaction with the virtual environment to ensure the virtual reality of the hand and virtual hand consistency.
In order to set up the above system, this article has done the following research: Using RealSense depth camera to identify 22 bones of human body. Based on the physical structure of human hand, the characteristics of skeletal movement and the virtual hand hierarchy model, a virtual hand skeleton model that is consistent with human hand characteristics and real-time and consistent is established by using Unity3D software. In order to realize the human-computer interaction under the virtual environment, this paper writes a script under the Unity3D platform to complete the design of the VR virtual numeric keypad and realize the virtual hand gesture interaction technology of digitizing the fingertip by pressing the numeric keys. Among them, in order to achieve a more accurate fingertip click operation, this paper detects, locates and tracks the fingertip through the key point detection of the fingertip to achieve the accurate positioning of the fingertip of the fingertip. The specific implementation method is to detect the contour of the finger contour after cutting and translating, detect the initial state fingertip point by using the curvature relationship of the hand side pixel, and perform fitting optimization and space coordinate conversion to
哈尔滨工业大学工程硕士学位论文
obtain the check point. Finally, Kalman filter is used to get the accurate three-dimensional coordinates of fingertip. Experiments show that the virtual hand model has a good effect and high real-time performance. Compared with the traditional method, the finger-tip detection part has less error with the real fingertip data and has higher robustness, and the fingertips click accuracy rate above 95%.
Keywords: augmented reality; virtual hands skeleton model; fingertip detection;
RealSense; unity3D
目录
摘要................................................................................................................. I Abstract .............................................................................................................. I I 第1章绪论 .. (1)
1.1课题来源 (1)
1.2课题研究的目的及意义 (2)
1.3国内外研究现状及分析 (2)
1.4本文主要研究内容 (4)
第2章建立虚拟手模型的相关理论基础 (6)
2.1引言 (6)
2.2人手骨骼结构原理 (6)
2.3虚拟手层次结构模型原理 (8)
2.3.1 虚拟手的层次结构模型 (8)
2.3.2 拇指运动模型及约束条件 (9)
2.3.3 四指运动模型及约束条件 (9)
2.4虚拟手运动映射原理 (12)
2.4.1 本文采用的运动映射方法 (12)
2.4.2 从人手到虚拟手的运动映射 (13)
2.5指尖跟踪中的卡尔曼滤波 (15)
2.5.1 KF指尖跟踪原理 (15)
2.5.2 KF的参数定义 (17)
2.6指尖检测相关原理 (18)
2.6.1 指尖模板匹配法 (18)
2.6.2 径向对称变换法 (19)
2.6.3 最大距离阈值法 (19)
2.6.4 轮廓曲率法 (19)
2.7本章小结 (21)
第3章虚拟手模型及应用系统设计 (22)
3.1引言 (22)
3.2虚拟手模型的整体方案 (22)
3.3初步模型建立及骨骼深度数据采集 (22)
3.3.1 初步模型建立 (23)
3.3.2 骨骼深度数据采集 (24)
3.4虚拟手模型建立算法 (26)
3.4.1 四指模型建立 (26)
3.4.2 拇指模型建立 (28)
3.4.3 坐标转换 (28)
3.5虚拟手姿态及映射实现 (30)
3.6虚拟手指尖检测概述及前期准备 (31)
3.6.1 指尖检测概述 (31)
3.6.2 手部分割 (32)
3.7虚拟手指尖特征点检测 (33)
3.7.1 目标人手及指尖水平变换 (34)
3.7.2 指尖特征点初始检测 (34)
3.7.3 指尖特征点的优化 (35)
3.7.4 指尖特征点跟踪 (35)
3.8本章小结 (36)
第4章虚拟手模型建立结果及性能分析 (37)
4.1引言 (37)
4.2系统软硬件环境 (37)
4.2.1 系统硬件环境 (37)
4.2.2 系统软件环境 (38)
4.2.3 虚拟手骨骼模型界面 (38)
4.3虚拟手模型结果评估 (39)
4.4虚拟键盘界面应用与交互 (41)
4.4.1 虚拟数字键盘界面搭建 (41)
4.4.2 虚拟键盘的逻辑判断 (42)
4.4.3 指尖触发判断 (43)
4.5虚拟手模型指尖运动结果评估 (44)
4.5.1 指尖特征点检测分析 (44)
4.5.2 指尖数字输入结果评估 (46)
4.6本章小结 (47)
结论 (48)
参考文献 (50)
攻读硕士学位期间发表的学术论文 (54)
哈尔滨工业大学学位论文原创性声明及使用授权说明 (55)
致谢 (56)