arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.06107cs.CLcs.AI
收藏

Doppelgänger's Watch:一种面向大语言模型的分离目标方法

Doppelgänger's Watch: A Split Objective Approach to Large Language Models

Meta Reality Labs(元宇宙实验室);Meta Gen AI(元通用人工智能部门)

Shervin Ghasemlou, Ashish Katiyar, Aparajita Saraf, Seungwhan Moon, Mangesh Pujari, Pinar Donmez, Babak Damavandi, Anuj Kumar

AI总结:

针对大语言模型的生成监督问题,提出并行模块 Doppelgänger 及两院制分离目标架构,实现逐 token 监督与评分预测,本文主要给出理论结果。

摘要:

本文研究大语言模型中的“生成监督”问题,并提出一种新颖的两院制架构,将监督信号与其核心能力(有用性)分离开来。Doppelgänger 是一个与底层语言模型并行的新模块,用于监督每个 token 的生成,并学习同时预测截至并包含每个 token 的序列的监督分数。在本工作中,我们呈现理论发现,而实验结果报告留待即将发表的出版物。

英文摘要:

In this paper, we investigate the problem of "generation supervision" in large language models, and present a novel bicameral architecture to separate supervision signals from their core capability, helpfulness. Doppelgänger, a new module parallel to the underlying language model, supervises the generation of each token, and learns to concurrently predict the supervision score(s) of the sequences up to and including each token. In this work, we present the theoretical findings, and leave the report on experimental results to a forthcoming publication.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑