NoRA: 评估视觉第一人称规范性动作推理中的基于事实的合理性
NoRA: Evaluating Grounded Reasonableness in Visual First-person Normative Action Reasoning
AI总结：
提出NoRA基准，通过事实-理由-动作支持图评估多模态模型生成合理动作并基于可见事实进行推理的能力，发现当前VLM在构建完整动作空间和绑定正确支持方面存在不足。
摘要：
LLM和智能系统越来越多地部署在社交环境中，使得规范能力对安全和适当行为至关重要。然而，现有方法要么仅在文本中评估规范性判断，要么将其简化为从固定候选动作集中选择。我们认为两者都不够。在实践中，智能体永远不会获得一个选项菜单；它们必须从头识别一个合理的动作，基于可见事实并由可检查的理由支持。我们引入了NoRA，一个视觉第一人称视频基准，要求模型生成候选的下一个动作，并通过显式的事实-理由-动作支持图来证明每个动作。该基准包含1,420个带注释的视频片段，包括HumanGold-190和LLMSilver-1230分割。每个实例通过动作对齐、事实基础和支持绑定进行评估，汇总为单一的基于事实的合理性分数。我们在直接、深思熟虑和结构化提示模式下对12个多模态系统进行了基准测试，发现当前的VLM经常能恢复合理的动作和相关的场景事实，但始终难以构建完整的合理动作空间并将所选动作绑定到正确的局部支持上。NoRA使这一差距可测量，将评估问题从模型是否能选择一个动作转变为是否能基于正确的可见理由证明一个适当的动作。
英文摘要：
LLMs and agentic systems are increasingly deployed in social environments, making normative competence critical for safe and appropriate behavior. However, existing approaches either assess normative judgment in text alone or reduce it to choosing among a fixed set of candidate actions. We argue both are insufficient. In practice, agents are never handed a menu of options; they must identify a reasonable action from scratch, grounded in visible facts and supported by inspectable reasons. We introduce NoRA, a visual first-person video benchmark that requires models to generate candidate next actions and justify each through an explicit fact-reason-action support graph. The benchmark comprises 1,420 annotated video clips, including HumanGold-190 and LLMSilver-1230 splits. Each instance is evaluated through action alignment, factual grounding, and support binding, aggregated into a single grounded reasonableness score. We benchmark 12 multimodal systems under direct, deliberate, and structured prompting regimes, finding that current VLMs frequently recover plausible actions and relevant scene facts, but consistently struggle to construct the full reasonable action space and bind selected actions to the correct local support. NoRA makes this gap measurable, shifting the evaluation question from whether a model can pick an action to whether it can justify an appropriate action for the right visible reasons.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
