EgoNormia: 物理社会规范理解基准测试
EgoNormia: Benchmarking Physical Social Norm Understanding
AI总结：
本研究提出基于第一人称交互视频的物理社会规范理解基准EGONORMIA，构建大规模数据集生成 pipeline，发现当前SOTA视觉语言模型规范理解能力不足，并验证了基于EGONORMIA的RAG方法可提升其规范性推理能力。
摘要：
人类活动受到规范的约束，但针对规范性推理的监督数据十分稀缺，尤其是在规范基于物理或社会场景的领域。为此，我们提出EGONORMIA $\|ε\
英文摘要：
Human activity is moderated by norms; however, supervision for normative reasoning is sparse, particularly where norms are physically- or socially-grounded. We thus present EGONORMIA $\|ε\|$, comprising 1,853 (200 for EGONORMIA-verified) multiple choice questions (MCQs) grounded within egocentric videos of human interactions, enabling the evaluation and improvement of normative reasoning in vision-language models (VLMs). EGONORMIA spans seven norm categories: safety, privacy, proxemics, politeness, cooperation, coordination/proactivity, and communication/legibility. To compile this dataset at scale, we propose a novel pipeline to generate grounded MCQs from raw egocentric video. Our work demonstrates that current state-of-the-art VLMs lack robust grounded norm understanding, scoring a maximum of 54% on EGONORMIA and 65% on EGONORMIA-verified, with performance across norm categories indicating significant risks of safety and privacy when VLMs are used in real-world agents. We additionally explore methods for improving normative understanding, demonstrating that a naive retrieval-based generation (RAG) method using EGONORMIA can enhance normative reasoning in VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
