面向开放世界第一人称活动识别的概率跳跃-扩散框架
A Probabilistic Jump-Diffusion Framework for Open-World Egocentric Activity Recognition
AI总结：
提出基于跳跃-扩散的概率残差搜索框架ProbRes，通过结合常识先验与VLM优化，在多开放度级别下高效识别未见第一人称活动并达到SOTA性能。
摘要：
开放世界第一人称活动识别因其无约束性质构成根本挑战，要求模型从广阔且部分观测的搜索空间中推断未见活动。我们提出ProbRes，一个基于跳跃-扩散的概率残差搜索框架，通过平衡先验引导的探索与似然驱动的利用来高效导航该空间。该方法整合结构化常识先验以构建语义连贯的搜索空间，利用视觉语言模型（VLMs）自适应优化预测，并采用随机搜索机制定位高似然活动标签，同时高效减少穷举枚举。我们在多个开放度级别（L0--L3）系统评估ProbRes，证明其对搜索空间复杂性增加的适应能力。除了在基准数据集（GTEA Gaze、GTEA Gaze+、EPIC-Kitchens和Charades-Ego）上取得最先进性能外，我们还建立了清晰的开放世界识别分类体系，阐明第一人称活动理解所需的挑战与方法论进步。
英文摘要：
Open-world egocentric activity recognition poses a fundamental challenge due to its unconstrained nature, requiring models to infer unseen activities from an expansive, partially observed search space. We introduce ProbRes, a Probabilistic Residual search framework based on jump-diffusion that efficiently navigates this space by balancing prior-guided exploration with likelihood-driven exploitation. Our approach integrates structured commonsense priors to construct a semantically coherent search space, adaptively refines predictions using Vision-Language Models (VLMs) and employs a stochastic search mechanism to locate high-likelihood activity labels while minimizing exhaustive enumeration efficiently. We systematically evaluate ProbRes across multiple openness levels (L0--L3), demonstrating its adaptability to increasing search space complexity. In addition to achieving state-of-the-art performance on benchmark datasets (GTEA Gaze, GTEA Gaze+, EPIC-Kitchens, and Charades-Ego), we establish a clear taxonomy for open-world recognition, delineating the challenges and methodological advancements necessary for egocentric activity understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
