arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.08604cs.CLcs.AI
收藏

NormSAGE:从对话中即时发现多语言多文化规范

NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Fly

University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Columbia University(哥伦比亚大学);Stony Brook University(石溪大学)

Yi R. Fung, Tuhin Chakraborty, Hao Guo, Owen Rambow, Smaranda Muresan, Heng Ji

AI总结:

NormSAGE利用GPT-3提示与自验证机制实现多语言多文化对话规范的即时发现,显著提升规范相关性与洞察力,并支持可解释的规范遵守与违反实例化。

摘要:

规范发现对于理解和推理人类交流与互动中的可接受行为及潜在违规具有重要意义。我们提出NormSAGE,一个基于语言模型提示与自验证机制、用于解决以对话为基础的多语言多文化规范发现这一新任务的框架。NormSAGE利用预训练GPT-3语言模型主干的表达能力和隐式知识,通过表示规范发现任务和对话上下文的定向提问来引出关于规范的知识。该框架进一步通过自验证机制应对语言模型幻觉风险,确保所发现的规范是正确的,并且实质性地扎根于其来源对话。评估结果表明,与基线相比,我们的方法能够即时发现显著更相关且更具洞察力的对话规范(Likert量表评分提升超过10%)。从中文对话中发现的规范在洞察力和正确性方面也与从英文对话中发现的规范相当(差异小于3%)。此外,文化特定规范具有良好质量,在文化配对人工识别中达到80%的准确率。最后,我们在规范发现自验证中的扎根过程可以扩展用于即时实例化给定对话中任何规范的遵守与违反情况,并具备可解释性和透明性。NormSAGE在扎根任务中实现了95.4%的AUC,其自然语言解释与人工撰写的质量相当。

英文摘要:

Norm discovery is important for understanding and reasoning about the acceptable behaviors and potential violations in human communication and interactions. We introduce NormSage, a framework for addressing the novel task of conversation-grounded multi-lingual, multi-cultural norm discovery, based on language model prompting and self-verification. NormSAGE leverages the expressiveness and implicit knowledge of the pretrained GPT-3 language model backbone, to elicit knowledge about norms through directed questions representing the norm discovery task and conversation context. It further addresses the risk of language model hallucination with a self-verification mechanism ensuring that the norms discovered are correct and are substantially grounded to their source conversations. Evaluation results show that our approach discovers significantly more relevant and insightful norms for conversations on-the-fly compared to baselines (>10+% in Likert scale rating). The norms discovered from Chinese conversation are also comparable to the norms discovered from English conversation in terms of insightfulness and correctness (<3% difference). In addition, the culture-specific norms are promising quality, allowing for 80% accuracy in culture pair human identification. Finally, our grounding process in norm discovery self-verification can be extended for instantiating the adherence and violation of any norm for a given conversation on-the-fly, with explainability and transparency. NormSAGE achieves an AUC of 95.4% in grounding, with natural language explanation matching human-written quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑