中国手语单手词汇识别方法和技术研究

CORC > 自动化研究所 > 中国科学院自动化研究所 > 毕业生 > 博士学位论文

题名	中国手语单手词汇识别方法和技术研究
作者	邹伟
学位类别	工学博士
答辩日期	2003-06-01
授予单位	中国科学院研究生院
授予地点	中国科学院自动化研究所
导师	原魁
关键词	中国手语模式识别数据手套肘部弯曲传感器模糊推理 FSMM 特征匹配 D-S证据理论 Chinese Sign Language Pattern Recognition Dataglove Sensor Fixed on Elbow Fuzzy Reasoning Feature Matching D-S Evidence Theory
其他题名	Research on Recognition Methods and Technologies of Single Hand Words of Chinese Sign Language
学位专业	控制理论与控制工程
中文摘要	手语处理技术是一种新型的应用技术。计算机手语识别研究的目的是使计算机能够正确理解听障者给出的手语信息，并给出相应的文本、语音等输出，从而达到帮助正常人与听障者进行交流的目的。本文在国家863项目“高性能、低成本数据手套的研制”的资助下，对中国手语中的单手词汇进行了分析，并在根据手姿信息和手势信息从识别的角度对该类词进行有效分类的基础上，借助于模糊思想对单手词汇的识别方法进行了探讨和研究。本论文的主要创新性工作总结如下： 1．结合肘部传感器的应用特点，提出了一种基于模糊推理的传感器建模方法：首先利用模糊推理建立传感器的模型，然后利用采样数据进行离线学习确定模型中的参数。实验证明该方法在只获取传感器最小和最大两个标定值的情况下，能够取得和分段线性插值相近似的结果。 2．对人手的空间运动进行了分析和建模。在此基础上提出了一种利用双信息源进行人手空间定位和跟踪的新方法：利用视觉输入和色彩模型提取人手的平面位置信息，借助于安装于肘部的弯曲传感器获取手臂的弯曲角度，根据所建模型由二者通过模糊计算确定人手的空间信息。实验证实了该方法的有效性。 3．提出了一种基于模糊神经网络的单手静态词识别方法：首先利用经验知识为每个词汇创建模糊规则，确定RBF网络的连接结构，然后通过学习确定各模糊子集隶属函数中的参数。对于参数的学习，提出了一种适用于分类器的可微经验风险函数，该函数能够有效地利用梯度下降法进行最小化。在实验中，根据识别率和平均可信度两个性能指标将本方法同基于聚类的模糊推理方法和基于特征匹配的方法进行。比较，证实了该方法的有效性和可靠性。 4．结合中国手语中手势的特点，采用人手的三维空间位置信息作为观测向量，将模糊理论与HMM理论相结合，提出了一种基于FSMM的中国手语手势识别方法。利用观测向量相对于各状态的隶属度对Baum-Welch算法进行了改进，实现了FSMM学习递推算法和识别算法。 5．利用数据手套、视觉设备和肘部弯曲传感器作为输入设备，提出了一种基于特征匹配和信息融合的中国手语单手词汇识别系统。首先利用手姿信息之间的相似性度量和手势信息之间的平均交叉熵来确定待识词所属类别并选择相应的词库；然后根据所属类别将待识词和库中词的相应特征按一定的规则进行匹配；最后利用D-S证据理论将各特征的匹配结果进行融合，选择具有最大基本概率赋值的词汇作为输出。实验结果表明，该
英文摘要	The technology of sign language processing is a practical technology around the world. The aim of the recognition of sign language is to provide an efficient and accurate mechanism to translate human sign language into text or speech, which can help deaf people to hear society communication. This thesis is supported by "863" national high-tech project "Researching and Developing a Kind of High Performance and Low Cost Dataglove". From the viewpoint of recognition, the single-hand words of CSL(Chinese Sign Language ) are analyzed and categorized based on the information of hand-pose and gesture. According to the features and categories of CSL words, the recognition methods are discussed and researched based on the fuzzy idea. The novel work and contribution of this thesis can be summarized as follows: 1. The characteristic of sensors fixed on the elbow is analyzed and a novel method of sensor modeling is proposed, which is based on the fuzzy reasoning. First, the model of sensors is established by use of the fuzzy reasoning, then the parameters are obtained by the off-line learning. The experiments show that this method can reach the target which the piecewise linear interpolation can meet, but only needs two gauge points: the maximum value and the minimum value of the sensors. 2. The 3D space motion of hand is analyzed and the model about it is built. A new method is introduced, which can locate the 3D position of hand by dual information sources: get the plane position by computer vision and color model of hand, get the angle of elbow by sensor. Based on these and the model built, the distance from hand to body can be obtained by fuzzy number computation. This method is confirmed to be feasible by experiments. 3. A novel recognition method of single-hand static words of CSL based on the fuzzy neural network is introduced. First, the fuzzy reasoning rules and the structure of the RBF (Radial Basis Function) network are established based on the empirical knowledge. Then the parameters in member function of each fuzzy subsets are obtained by learning. During the process of learning, a new kind of empirical risk function is proposed which is differentiable and can be minimized by gradient descent strategy. In experiments, this method is compared with the method of fuzzy reasoning based on clustering and the method of feature matching in recognition rate and average degree of belief, so the validity and reliability of this method are confirmed. 4. According to the features of CSL gestures, a novel recognition method based on the FSMM(Fuzzy State Markov Model ) is introduced, which combines the fuzzy theory and HMM(Hidden Markov Model) theory. In this method, the 3D position of hand is adopted as the observable vector. The algorithm of Baum-Welch is improved by use of the membership function of each state. The recursive learning algorithm of FSMM and recognition algorithm are implemented. 5. A recognition system
语种	中文
其他标识符	719
内容类型	学位论文
源URL	[http://ir.ia.ac.cn/handle/173211/5767]
专题	毕业生_博士学位论文
推荐引用方式 GB/T 7714	邹伟. 中国手语单手词汇识别方法和技术研究[D]. 中国科学院自动化研究所. 中国科学院研究生院. 2003.