arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.11960cs.CV
收藏

基于复杂环境的中国连续手语数据集

A Chinese Continuous Sign Language Dataset Based on Complex Environments

College of Intelligent Systems Science and Engineering(智能系统科学与工程学院);Harbin Engineering University(哈尔滨工程大学);Northwest Institute of Mechanical and Electrical Engineering(西北机电工程研究院);Kyungil University(庆北大学)

Qidan Zhu, Jing Li, Fei Yuan, Jiaojiao Fan, Quan Gan

AI总结:

针对现有连续手语识别数据集背景单一的问题,构建了含5,988个视频、70余种复杂背景的CE-CSL数据集,并提出融合时频特征的TFNet模型,在复杂背景及公开数据集上均取得显著性能提升。

摘要:

当前连续手语识别(CSLR)研究的瓶颈在于,大多数公开数据集仅限于实验室环境或电视节目录制,导致背景环境单一、光照均匀,这与现实生活场景中的多样性和复杂性严重脱节。为应对这一挑战,我们构建了一个基于复杂环境的、新的大规模中国连续手语(CSL)数据集,称为复杂环境-中国手语数据集(CE-CSL)。该数据集包含从日常生活场景中采集的5,988个连续手语视频片段,涵盖超过70种不同的复杂背景,以确保代表性和泛化能力。为解决复杂背景对CSLR性能的影响,我们提出了一种用于连续手语识别的时频网络(TFNet)模型。该模型提取帧级特征,然后利用时域和频域信息分别推导序列特征,再进行融合,旨在实现高效且准确的CSLR。实验结果表明,我们的方法在CE-CSL上取得了显著的性能提升,验证了其在复杂背景条件下的有效性。此外,我们提出的方法在三个公开的CSL数据集上也取得了极具竞争力的结果。

英文摘要:

The current bottleneck in continuous sign language recognition (CSLR) research lies in the fact that most publicly available datasets are limited to laboratory environments or television program recordings, resulting in a single background environment with uniform lighting, which significantly deviates from the diversity and complexity found in real-life scenarios. To address this challenge, we have constructed a new, large-scale dataset for Chinese continuous sign language (CSL) based on complex environments, termed the complex environment - chinese sign language dataset (CE-CSL). This dataset encompasses 5,988 continuous CSL video clips collected from daily life scenes, featuring more than 70 different complex backgrounds to ensure representativeness and generalization capability. To tackle the impact of complex backgrounds on CSLR performance, we propose a time-frequency network (TFNet) model for continuous sign language recognition. This model extracts frame-level features and then utilizes both temporal and spectral information to separately derive sequence features before fusion, aiming to achieve efficient and accurate CSLR. Experimental results demonstrate that our approach achieves significant performance improvements on the CE-CSL, validating its effectiveness under complex background conditions. Additionally, our proposed method has also yielded highly competitive results when applied to three publicly available CSL datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑