arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.00210cs.CLcs.CY
收藏

对齐问题

The Problem of Alignment

Winchester School of Art, University of Southampton(南安普顿大学温彻斯特艺术学院);Institute for Culture and Society, Western Sydney University(西悉尼大学文化与社会研究所);Toronto Metropolitan University(多伦多都会大学)

Tsvetelina Hristova, Liam Magee, Karen Soldatic

AI总结:

本文探讨大型语言模型的对齐问题,将其视为规范结构在统计模型上的叠加,通过分析 ChatGPT4 对乔伊斯文本的编辑及提示工程,结合战后语言学辩论与莫斯科语言学派,揭示用户与模型间的互动及社会结构化影响。

摘要:

大型语言模型生成从大型语料库中学习到的统计模式序列。为了不复制语料库偏差,在初始训练之后,模型必须与人类价值观对齐,偏好某些续写而非其他。对齐可以被视为将规范结构叠加到统计模型上,它揭示了语言与技术之间冲突且复杂的相互关系。这种关系塑造了语言理论、语言实践和主观性,这与当前人工生成文本的复杂性尤为相关。我们通过分析 ChatGPT4 如何编辑乔伊斯《尤利西斯》片段中被视为“异常”的语言以及提示工程这一新的语言实践,将这种结构化实践作为用户与模型之间的双向互动进行考察。然后,我们在历史上定位这一对齐问题,回顾战后早期的语言学辩论,这些辩论将两种意义观对立起来:作为离散结构的意义,以及作为连续概率分布的意义。我们讨论了莫斯科语言学派在很大程度上被遮蔽的工作,该学派试图调和这一对立。我们对莫斯科学派以及 Searle 和 Kristeva 后来相关论点的关注,为对齐问题投下了新的视角:作为一个涉及关注语言实践社会结构化的问题,包括对异常的结构化,这些异常像乔伊斯文本一样,公然违抗表达惯例。这些围绕语言交际导向的辩论有助于解释用户与 LLMs 之间发生的一些当代行为和相互依赖关系。

英文摘要:

Large Language Models produce sequences learned as statistical patterns from large corpora. In order not to reproduce corpus biases, after initial training models must be aligned with human values, preferencing certain continuations over others. Alignment, which can be viewed as the superimposition of normative structure onto a statistical model, reveals a conflicted and complex interrelationship between language and technology. This relationship shapes theories of language, linguistic practice and subjectivity, which are especially relevant to the current sophistication in artificially produced text. We examine this practice of structuration as a two-way interaction between users and models by analysing how ChatGPT4 redacts perceived `anomalous' language in fragments of Joyce's Ulysses and the new linguistic practice of prompt engineering. We then situate this alignment problem historically, revisiting earlier postwar linguistic debates which counterposed two views of meaning: as discrete structures, and as continuous probability distributions. We discuss the largely occluded work of the Moscow Linguistic School, which sought to reconcile this opposition. Our attention to the Moscow School and later related arguments by Searle and Kristeva casts the problem of alignment in a new light: as one involving attention to the social structuration of linguistic practice, including structuration of anomalies that, like the Joycean text, exist in defiance of expressive conventions. These debates around the communicative orientation toward language can help explain some of the contemporary behaviours and interdependencies that take place between users and LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑