arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.04003cs.AI
收藏

当AI说服人:对抗性解释攻击对人类信任AI辅助决策的影响

When AI Persuades: Adversarial Explanation Attacks on Human Trust in AI-Assisted Decision Making

Clemson University(克莱姆森大学)

Shutong Fan, Lan Zhang, Xiaoyong Yuan

AI总结:

本文研究了对抗性解释攻击如何通过操控LLM生成的解释框架,影响人类对AI输出的信任,揭示了认知层的新型安全风险。

摘要:

大多数对抗性威胁针对AI模型的计算行为,而非依赖它们的人类。然而,现代AI系统越来越多地在人类决策循环中运行,用户根据模型推荐进行解释和行动。大型语言模型(LLMs)生成流畅的自然语言解释,影响用户对AI输出的认知和信任,揭示了AI与用户之间的沟通渠道这一新攻击面。我们引入对抗性解释攻击(AEAs),攻击者操控LLM生成的解释框架以调节人类对错误输出的信任。我们通过信任失调差距这一指标,正式化这一行为威胁,该指标捕捉了良性与对抗性解释之间人类信任的差异。通过这一指标,我们强调了说服性解释框架可能在AI预测错误时仍能保持用户信任的行为风险。为了表征这一威胁,我们进行了包含超过200名参与者的实验,系统地变化解释框架的四个维度:推理模式、证据类型、沟通风格和呈现格式。我们的发现显示,用户对对抗性和良性解释的信任几乎相同,对抗性解释尽管错误,却保留了大多数良性信任。最脆弱的情况出现在AEAs接近专家沟通时,结合权威证据、中性语气和领域合适的推理。脆弱性最高出现在困难任务、事实驱动领域以及受教育程度较低、年轻或高度信任AI的参与者中。

英文摘要:

Most adversarial threats in artificial intelligence (AI) target the computational behavior of models rather than the humans who rely on them. Yet modern AI systems increasingly operate within human decision loops, where users interpret and act on model recommendations. Large Language Models (LLMs) generate fluent natural-language explanations that shape how users perceive and trust AI outputs, revealing a new attack surface at the cognitive layer: the communication channel between AI and its users. We introduce adversarial explanation attacks (AEAs), where an attacker manipulates the framing of LLM-generated explanations to modulate human trust in incorrect outputs. We formalize this behavioral threat through the trust miscalibration gap, a metric that captures the difference in human trust between benign and adversarial explanations. Using this metric as a lens, we highlight a behavioral risk where persuasive explanation framing can preserve user trust even when the underlying AI prediction is wrong. To characterize this threat, we conducted a human study with over 200 participants, systematically varying four dimensions of explanation framing: reasoning mode, evidence type, communication style, and presentation format. Our findings show that users report nearly identical trust for adversarial and benign explanations, with adversarial explanations preserving the vast majority of benign trust despite being incorrect. The most vulnerable cases arise when AEAs closely resemble expert communication, combining authoritative evidence, neutral tone, and domain-appropriate reasoning. Vulnerability is highest on hard tasks, in fact-driven domains, and among participants who are less formally educated, younger, or highly trusting of AI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑