具备可靠性和安全性的人工智能(DAIReS):一种统一的综合解码方法用于幻觉和后门触发检测
Dependable Artificial Intelligence with Reliability and Security (DAIReS): A Unified Syndrome Decoding Approach for Hallucination and Backdoor Trigger Detection
AI总结:
DAIReS提出基于综合解码的统一方法,用于检测学习系统中的安全和可靠性违规,包括后门攻击和幻觉检测。
摘要:
机器学习(ML)模型,包括大语言模型(LLMs),具有诸如安全性和可靠性等系统级属性。最近的研究表明,ML模型容易受到多种安全违规形式的攻击,其中后门数据中毒攻击代表了特别危险的威胁,能够导致未经授权的模型行为和系统性误分类。同时,模型可靠性缺陷可能在LLMs中表现为幻觉,导致不可预测的输出和对终端用户造成重大风险。在本次关于具备可靠性和安全性的AI(DAIReS)的研究中,我们提出了一种基于综合解码的新型统一方法,用于检测基于学习系统的安全性和可靠性违规。具体来说,我们将综合解码方法适应到NLP句子嵌入空间,使得能够在ML训练数据集中区分受污染和非受污染的样本。此外,相同的方法可以有效检测由于自指元解释任务导致的幻觉内容。
英文摘要:
Machine Learning (ML) models, including Large Language Models (LLMs), are characterized by a range of system-level attributes such as security and reliability. Recent studies have demonstrated that ML models are vulnerable to multiple forms of security violations, among which backdoor data-poisoning attacks represent a particularly insidious threat, enabling unauthorized model behavior and systematic misclassification. In parallel, deficiencies in model reliability can manifest as hallucinations in LLMs, leading to unpredictable outputs and substantial risks for end users. In this work on Dependable Artificial Intelligence with Reliability and Security (DAIReS), we propose a novel unified approach based on Syndrome Decoding for the detection of both security and reliability violations in learning-based systems. Specifically, we adapt the syndrome decoding approach to the NLP sentence-embedding space, enabling the discrimination of poisoned and non-poisoned samples within ML training datasets. Additionally, the same methodology can effectively detect hallucinated content due to self referential meta explanation tasks in LLMs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。