arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.09254cs.LGcs.CLstat.ML
收藏

面向可控语言模型的选择性生成

Selective Generation for Controllable Language Models

GSAI POSTECH(浦项科技大学人工智能研究生院);SCS & SCP GaTech(佐治亚理工学院计算机科学学院与计算科学与工程学院);CSE POSTECH(浦项科技大学计算机科学与工程学院)

Minjae Lee, Kyungmin Kim, Taesoo Kim, Sangdon Park

AI总结:

本文提出两种选择性生成算法SGenSup和SGenSemi,利用文本蕴含评估生成正确性,在理论保证下控制错误发现率,并通过半监督伪标记提升选择效率。

摘要:

生成式语言模型(GLMs)在部署到关键决策系统时的可信度至关重要。因此,诸如选择性预测和保形预测等经认证的风险控制方法已被用于缓解各种有监督下游任务中的幻觉问题。然而,缺乏合适的正确性度量指标阻碍了将此类有原则的方法应用于语言生成任务。在本文中,我们通过利用文本蕴含的概念来评估生成序列的正确性,从而规避了这一问题,并提出了两种选择性生成算法,它们在理论保证下控制关于文本蕴含关系的错误发现率(FDR-E):$\ exttt{SGen}^{\ exttt{Sup}}$和$\ exttt{SGen}^{\ exttt{Semi}}$。$\ exttt{SGen}^{\ exttt{Sup}}$是对选择性预测的直接修改,是一种利用由人类标注的蕴含标签数据的有监督学习算法。由于人工标注成本高昂,我们进一步提出了半监督版本$\ exttt{SGen}^{\ exttt{Semi}}$,它通过伪标记充分利用未标记数据,并利用通过保形预测学习的蕴含集函数。此外,$\ exttt{SGen}^{\ exttt{Semi}}$能够使用更通用的选择函数类,即神经选择函数,并在给定多个候选者时为用户提供最优的选择函数类。最后,我们证明了$\ exttt{SGen}$家族在实现所需FDR-E水平方面的有效性,其在开源和闭源GLM上的选择效率与基线方法相当。代码和数据集可在https://github.com/ml-postech/selective-generation获取。

英文摘要:

Trustworthiness of generative language models (GLMs) is crucial in their deployment to critical decision making systems. Hence, certified risk control methods such as selective prediction and conformal prediction have been applied to mitigating the hallucination problem in various supervised downstream tasks. However, the lack of appropriate correctness metric hinders applying such principled methods to language generation tasks. In this paper, we circumvent this problem by leveraging the concept of textual entailment to evaluate the correctness of the generated sequence, and propose two selective generation algorithms which control the false discovery rate with respect to the textual entailment relation (FDR-E) with a theoretical guarantee: $\texttt{SGen}^{\texttt{Sup}}$ and $\texttt{SGen}^{\texttt{Semi}}$. $\texttt{SGen}^{\texttt{Sup}}$, a direct modification of the selective prediction, is a supervised learning algorithm which exploits entailment-labeled data, annotated by humans. Since human annotation is costly, we further propose a semi-supervised version, $\texttt{SGen}^{\texttt{Semi}}$, which fully utilizes the unlabeled data by pseudo-labeling, leveraging an entailment set function learned via conformal prediction. Furthermore, $\texttt{SGen}^{\texttt{Semi}}$ enables to use more general class of selection functions, neuro-selection functions, and provides users with an optimal selection function class given multiple candidates. Finally, we demonstrate the efficacy of the $\texttt{SGen}$ family in achieving a desired FDR-E level with comparable selection efficiency to those from baselines on both open and closed source GLMs. Code and datasets are provided at https://github.com/ml-postech/selective-generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑