Spotter+GPT: Turning Sign Spottings into Sentences with LLMs
CVSSP;University of Surrey(萨里大学)
Ozge Mercanoglu Sincan, Richard Bowden
备注:
Accepted at the 9th Workshop on Sign Language Translation and Avatar Technologies (SLTAT) in ACM International Conference on Intelligent Virtual Agents (IVA`25)
Sign Language Translation (SLT) is a challenging task that aims to generate spoken language sentences from sign language videos. In this paper, we introduce a lightweight, modular SLT framework, Spotter+GPT, that leverages the power of Large Language Models (LLMs) and avoids heavy end-to-end training. Spotter+GPT breaks down the SLT task into two distinct stages. First, a sign spotter identifies individual signs within the input video. The spotted signs are then passed to an LLM, which transforms them into meaningful spoken language sentences. Spotter+GPT eliminates the requirement for SLT-specific training. This significantly reduces computational costs and time requirements. The source code and pretrained weights of the Spotter are available at https://gitlab.surrey.ac.uk/cogvispublic/sign-spotter.
Single Shot Self-Reliant Scene Text Spotter by Decoupled yet Collaborative Detection and Recognition
Harbin Institute of Technology at Shenzhen(哈尔滨工业大学(深圳));Baidu Inc(百度公司)
典型的文本识别遵循两阶段识别范式,首先检测文本实例的边界,然后在检测到的区域内执行文本识别。尽管这样的斑点范例的显着进步,一个重要的限制是,文本识别的性能在很大程度上取决于文本检测的精度,导致潜在的错误传播从检测到识别。在这项工作中,我们提出了Single Shot Self-Reliant Scene Text Spotter v2(SRSTS v2),它通过将识别与检测分离,同时协同优化两个任务来规避这一限制。具体来说,我们的SRSTS v2样本的代表性特征点周围的每个潜在的文本实例,并进行文本检测和识别并行引导这些采样点。因此,文本识别不再依赖于检测,从而减轻了从检测到识别的错误传播。此外,采样模块是在检测和识别的监督下学习的,这允许两个任务之间的协同优化和相互增强。受益于这种采样驱动的并发spotting框架,我们的方法是能够正确地识别文本实例,即使精确的文本边界是具有挑战性的检测。在四个基准上进行的大量实验表明,我们的方法与最先进的检举人相比毫不逊色。
GoMatching: A Simple Baseline for Video Text Spotting via Long and Short Term Matching
Institute of Artificial Intelligence, Wuhan University(武汉大学人工智能研究院);College of Computing & Data Science at Nanyang Technological University(南洋理工大学计算与数据科学学院)
GoMatching++: Parameter- and Data-Efficient Arbitrary-Shaped Video Text Spotting and Benchmarking
School of Computer Science, National Engineering Research Center for Multimedia Software, and Institute of Artificial Intelligence, Wuhan University(计算机学院、多媒体软件国家工程研究中心及人工智能学院,武汉大学)
TextBlockV2: Towards Precise-Detection-Free Scene Text Spotting with Pre-trained Language Model
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Lenovo Research(联想研究院);School of Cyber Science and Engineering, Nanjing University of Science and Technology(南京理工大学网络空间科学与工程学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
VimTS: A Unified Video and Image Text Spotter for Enhancing the Cross-domain Generalization
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(华中科技大学人工智能与自动化学院);South China University of Technology(华南理工大学);Zhejiang University(浙江大学)