PaperTrail: 一种用于在基于大语言模型的学术问答中建立溯源的声明-证据接口
PaperTrail: A Claim-Evidence Interface for Grounding Provenance in LLM-based Scholarly Q&A
AI总结：
PaperTrail通过分解LLM回答和源文档为声明与证据，帮助学术领域更严谨地验证LLM的可信度，并揭示其信任worthiness。
摘要：
大语言模型（LLMs）越来越多地用于学术问答（QA）系统，帮助研究人员整合大量文献。然而，这些系统常常会产生细微的错误（例如，未经支持的声明、遗漏的错误），而当前的溯源机制如来源引用对于学术领域所需的严格验证来说不够细致。为此，我们引入了PaperTrail，一种新的接口，将LLM的答案和源文档分解为离散的声明和证据，将其映射以揭示支持的断言、未经支持的声明以及从源文本中遗漏的信息。我们在26名研究者中进行了组内研究，他们使用PaperTrail和一个基线接口完成了两个学术编辑任务。我们的结果显示，与基线相比，PaperTrail显著降低了参与者的信任度。然而，这种增加的谨慎并未转化为行为改变，因为人们继续依赖LLM生成的学术编辑以避免认知负担的任务。我们讨论了在学术环境中声明-证据匹配的价值，以及对认知友好的溯源信息设计的启示。
英文摘要：
Large language models (LLMs) are increasingly used in scholarly question-answering (QA) systems to help researchers synthesize vast amounts of literature. However, these systems often produce subtle errors (e.g., unsupported claims, errors of omission), and current provenance mechanisms like source citations are not granular enough for the rigorous verification that scholarly domain requires. To address this, we introduce PaperTrail, a novel interface that decomposes both LLM answers and source documents into discrete claims and evidence, mapping them to reveal supported assertions, unsupported claims, and information omitted from the source texts. We evaluated PaperTrail in a within-subjects study with 26 researchers who performed two scholarly editing tasks using PaperTrail and a baseline interface. Our results show that PaperTrail significantly lowered participants' trust compared to the baseline. However, this increased caution did not translate to behavioral changes, as people continued to rely on LLM-generated scholarly edits to avoid a cognitively burdensome task. We discuss the value of claim-evidence matching for understanding LLM trustworthiness in scholarly settings, and present design implications for cognition-friendly communication of provenance information.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
