AI赋能的人类记忆操纵：误导性AI生成摘要扭曲人类记忆
AI-Enabled Human Memory Manipulation: Misleading AI-Generated Summaries Distort Human Memory
- Georgetown University（乔治城大学）
- University of Washington（华盛顿大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究通过AI摘要错误分析和人类实验，发现AI生成的误导性摘要会显著扭曲人类对原始事件的记忆，对高风险场景中AI的使用提出警示。
AI中文摘要：
AI生成的摘要越来越多地被用于高风险场景，如警务，尽管有大量证据表明AI经常生成误导性或不准確的信息。本研究提出疑问：AI生成摘要中的错误是否会扭曲人类记忆？为回答此问题，我们采用了两种方法论。首先，我们对AI摘要输出进行了分析，提示大型语言模型生成视频摘要。该分析量化了AI摘要包含错误的频率及错误类别，揭示了可能扭曲人类记忆的误导性信息种类。其次，我们进行了一项人类受试者实验，以测试AI生成摘要中误导性信息对人类记忆的影响。参与者首先通过观看一段汽车-行人事故视频接触事件，随后阅读一段描述该视频的AI生成摘要，该摘要包含误导性或准确信息。参与者在记忆识别测试中评估其对原始事件的记忆。在AI分析中，我们发现AI摘要中错误频率较高，尤其是关键细节的遗漏。例如，大多数摘要遗漏了视频最核心的事件，这是一个可能产生重大影响的关键错误。我们还观察到AI错误信息对人类记忆的强烈影响。与阅读准确AI摘要的人相比，阅读误导性AI摘要的人准确回忆原始事件的可能性显著降低。这些发现对AI在关键场景中的使用具有启示意义。尽管“人在回路”常被期望纠正AI的错误，但我们的研究表明，人类记忆反而可能被这些错误扭曲。AI有可能生成错误信息，即使没有任何对抗意图，也能对人类记忆产生实质性影响。
英文摘要：
AI-generated summaries are increasingly used in high-stakes settings, like policing, despite considerable evidence that AI often generates misleading or inaccurate information. This research asked: do errors in AI-generated summaries distort human memory? To answer this question, we adopted two methodological approaches. First, we conducted an analysis of AI summary output, prompting large language models to generate summaries of videos. This analysis quantified how often AI summaries contain errors and the categories of these errors, revealing the kinds of misleading information that may distort human memory. Second, we conducted a human-subjects experiment to test the impact of misleading information in AI-generated summaries on human memory. Participants were first exposed to an event via watching a video of a car-pedestrian accident, and later read an AI-generated summary describing the video that either contained misleading or accurate information. Participants' memory for the original event was assessed in a memory recognition test. In the AI analysis, we found a high frequency of mistakes in AI summaries, and particularly frequent omissions of critical details. For instance, the majority of summaries omitted the most central event of the video, a critical error which is likely to be impactful. We also observed a strong effect of AI misinformation on human memory. People who read a misleading AI summary were significantly less likely to accurately recall the original event, compared to people who read an accurate AI summary. These findings have implications for how AI should be used in critical settings. Though "humans-in-the-loop" are often expected to correct for AI's mistakes, our work suggests human memory can instead be distorted by these mistakes. AI has the potential to generate misinformation, even absent any adversarial intent, which can meaningfully impact human memory.
