arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.09667cs.CLcs.AIcs.LG
收藏

注视、归因与合理化:迈向物理安全且可信赖的AI

Foveate, Attribute, and Rationalize: Towards Physically Safe and Trustworthy AI

University of California, Santa Barbara(加州大学圣巴巴拉分校)

Alex Mei, Sharon Levy, William Yang Wang

AI总结:

针对隐蔽不安全文本难以检测的问题,提出FARM框架,通过注视缺失知识、归因检索可信赖外部信息,实现安全分类与可解释理由生成,在SafeText数据集上准确率提升5.9%。

摘要:

随着智能系统市场持续增长,用户的物理安全日益受到关注,不受约束的系统可能会向用户推荐危险行为,从而导致严重伤害。隐蔽不安全文本是特别受关注的领域,因为此类文本可能源于日常场景,且难以被检测出有害性。我们提出了FARM,一种利用外部知识在安全情境下生成可信赖理由的新颖框架。具体而言,FARM注视缺失的知识,以限定在特定场景下推理所需的信息,并通过归因至可信赖来源来检索这些信息。该知识既用于对原始文本的安全性进行分类,也用于生成人类可解释的理由,揭示系统对特定用户群体的风险,帮助利益相关者管理其系统风险,并帮助政策制定者为消费者安全提供具体保障。我们的实验表明,FARM在SafeText数据集上取得了最先进的结果,在安全分类准确率上实现了5.9%的绝对提升。

英文摘要:

Users' physical safety is an increasing concern as the market for intelligent systems continues to grow, where unconstrained systems may recommend users dangerous actions that can lead to serious injury. Covertly unsafe text is an area of particular interest, as such text may arise from everyday scenarios and are challenging to detect as harmful. We propose FARM, a novel framework leveraging external knowledge for trustworthy rationale generation in the context of safety. In particular, FARM foveates on missing knowledge to qualify the information required to reason in specific scenarios and retrieves this information with attribution to trustworthy sources. This knowledge is used to both classify the safety of the original text and generate human-interpretable rationales, shedding light on the risk of systems to specific user groups and helping both stakeholders manage the risks of their systems and policymakers to provide concrete safeguards for consumer safety. Our experiments show that FARM obtains state-of-the-art results on the SafeText dataset, showing absolute improvement in safety classification accuracy by 5.9%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑