arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.16672cs.CLcs.AI
收藏

CAVE:可控的作者身份验证解释

CAVE: Controllable Authorship Verification Explanations

University of Southern California(南加利福尼亚大学);Information Sciences Institute, University of Southern California(南加利福尼亚大学信息科学研究所)

Sahana Ramnath, Kartik Pandey, Elizabeth Boschee, Xiang Ren

AI总结:

本文提出离线模型 CAVE,通过生成结构统一且易于验证一致性的作者身份验证解释,结合 Prompt-CAVE 数据生成与 Cons-R-L 过滤,在保证任务准确率的同时提升解释质量。

摘要:

作者身份验证(AV)(判断两份文档是否出自同一作者)在许多现实应用中至关重要。AV 常用于隐私敏感领域,这些领域需要部署在本地的离线专有模型,因此公开服务的在线模型(API)并非最优选择。然而,当前的离线 AV 模型下游效用较低,原因在于准确率有限(例如传统文体计量学 AV 系统)以及缺乏可访问的事后解释。在本工作中,我们通过开发一个经过训练的离线模型 CAVE(Controllable Authorship Verification Explanations,可控作者身份验证解释)来应对上述挑战。CAVE 生成自由文本的 AV 解释,并控制其具备以下特性:(1)可访问性(具有统一结构,可分解为基于相关语言特征的子解释),以及(2)易于验证解释与标签之间的一致性。我们通过基于提示的方法 Prompt-CAVE,生成以所需语言特征为依据的银标准训练数据。随后,我们使用一种新的度量指标 Cons-R-L,基于理由-标签一致性对数据进行过滤。最后,我们利用这些数据对一个小型离线模型(Llama-3-8B)进行微调,从而构建出我们的模型 CAVE。在三个具有挑战性的 AV 数据集上的结果表明,CAVE 生成了高质量的解释(通过自动评估和人工评估衡量),并取得了具有竞争力的任务准确率。

英文摘要:

Authorship Verification (AV) (do two documents have the same author?) is essential in many real-life applications. AV is often used in privacy-sensitive domains that require an offline proprietary model that is deployed on premises, making publicly served online models (APIs) a suboptimal choice. Current offline AV models however have lower downstream utility due to limited accuracy (eg: traditional stylometry AV systems) and lack of accessible post-hoc explanations. In this work, we address the above challenges by developing a trained, offline model CAVE (Controllable Authorship Verification Explanations). CAVE generates free-text AV explanations that are controlled to be (1) accessible (uniform structure that can be decomposed into sub-explanations grounded to relevant linguistic features), and (2) easily verified for explanation-label consistency. We generate silver-standard training data grounded to the desirable linguistic features by a prompt-based method Prompt-CAVE. We then filter the data based on rationale-label consistency using a novel metric Cons-R-L. Finally, we fine-tune a small, offline model (Llama-3-8B) with this data to create our model CAVE. Results on three difficult AV datasets show that CAVE generates high quality explanations (as measured by automatic and human evaluation) as well as competitive task accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑