arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.16874cs.HCcs.AI
收藏

设计中的披露:对话式AI模型的身份透明性作为行为属性

Disclosure By Design: Identity Transparency as a Behavioural Property of Conversational AI Models

AI Security Institute(人工智能安全研究所);University of Southampton(萨默塞特大学)

Anna Gausen, Sarenne Wallbridge, Hannah Rose Kirk, Jennifer Williams, Christopher Summerfield

AI总结:

研究探讨对话式AI在人类与AI交互日益模糊时如何维持身份透明性,提出通过设计实现披露,以提升系统可靠性并保护用户自主验证能力。

摘要:

随着对话式AI系统变得更加真实并广泛部署,用户对交互对象是人类还是AI的不确定性增加。当AI身份不明确时,用户可能无意中分享敏感信息、对AI建议产生不合理的信任或成为AI欺诈的受害者。更广泛地说,持续缺乏透明度会削弱中介通信的信任。虽然如欧盟AI法案和加州BOT法案等法规要求AI系统自我标识,但它们对实时对话中的可靠披露提供有限指导。现有透明机制也存在漏洞:界面指示器可能被部署者省略,而溯源工具需要协调的基础设施且无法提供可靠的实时验证。我们探讨对话式AI系统应如何在人类与AI交互更加模糊和多样化时维持身份透明性。我们倡导通过设计实现披露,即当被直接询问时,AI系统明确披露其人工身份。实现为模型行为,披露可跨部署环境持续存在而无需依赖用户界面,同时保护用户自主验证身份的能力,而不干扰如角色扮演等沉浸式使用。为了评估当前实践,我们提出了首个多模态(文本和语音)评估,评估已部署系统在基线、角色扮演和对抗性设置中的披露行为。我们发现基线披露率通常较高,但在角色扮演中大幅下降,且在对抗性提示下可能被抑制。重要的是,披露率在不同提供商和模态间差异显著,突显了当前披露行为的脆弱性。我们最后提出技术干预,帮助开发者将披露作为对话式AI模型的基本属性。

英文摘要:

As conversational AI systems become more realistic and widely deployed, users are increasingly uncertain about whether they are interacting with a human or an AI system. When AI identity is unclear, users may unwittingly share sensitive information, place unwarranted trust in AI-generated advice, or fall victim to AI-enabled fraud. More broadly, a persistent lack of transparency can erode trust in mediated communication. While regulations like the EU AI Act and California's BOT Act require AI systems to identify themselves, they provide limited guidance on reliable disclosure in real-time conversation. Existing transparency mechanisms also leave gaps: interface indicators can be omitted by deployers, and provenance tools require coordinated infrastructure and cannot provide reliable real-time verification. We ask how conversational AI systems should maintain identity transparency as human-AI interactions become more ambiguous and diverse. We advocate for disclosure by design, where AI systems explicitly disclose their artificial identity when directly asked. Implemented as model behaviour, disclosure can persist across deployment contexts without relying on user interfaces, while preserving user agency to verify identity on demand without disrupting immersive uses like role-playing. To assess current practice, we present the first multi-modal (text and voice) evaluation of disclosure behaviour in deployed systems across baseline, role-playing, and adversarial settings. We find that baseline disclosure rates are often high but drop substantially in role-play and can be suppressed under adversarial prompting. Importantly, disclosure rates vary significantly across providers and modalities, highlighting the fragility of current disclosure behaviour. We conclude with technical interventions to help developers embed disclosure as a fundamental property of conversational AI models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑