Uncertainty Aware Learning for Language Model Alignment
School of Computer Science, Fudan University(复旦大学计算机科学学院);The University of Sydney(悉尼大学);The Chinese University of Hong Kong(香港中文大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Nanyang Technological University(南洋理工大学)
As instruction-tuned large language models (LLMs) evolve, aligning pretrained foundation models presents increasing challenges. Existing alignment strategies, which typically leverage diverse and high-quality data sources, often overlook the intrinsic uncertainty of tasks, learning all data samples equally. This may lead to suboptimal data efficiency and model performance. In response, we propose uncertainty-aware learning (UAL) to improve the model alignment of different task scenarios, by introducing the sample uncertainty (elicited from more capable LLMs). We implement UAL in a simple fashion -- adaptively setting the label smoothing value of training according to the uncertainty of individual samples. Analysis shows that our UAL indeed facilitates better token clustering in the feature space, validating our hypothesis. Extensive experiments on widely used benchmarks demonstrate that our UAL significantly and consistently outperforms standard supervised fine-tuning. Notably, LLMs aligned in a mixed scenario have achieved an average improvement of 10.62\% on high-entropy tasks (i.e., AlpacaEval leaderboard), and 1.81\% on complex low-entropy tasks (i.e., MetaMath and GSM8K).
UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models
The Chinese University of Hong Kong(香港中文大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);The University of Hong Kong(香港大学);The Hong Kong Polytechnic University(香港理工大学);Hong Kong University of Science and Technology, Guangzhou(香港科技大学(广州));MoE Key Laboratory of High Confidence Software Technologies(教育部高可信软件技术重点实验室)
Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China(计算与智能研究所,哈尔滨工业大学,深圳,中国);Peng Cheng Laboratory, Shenzhen, China(鹏城实验室,深圳,中国)
SURE: Synergistic Uncertainty-aware Reasoning for Multimodal Emotion Recognition in Conversations
Guangdong Provincial Key Laboratory of Quantum Engineering and Quantum Materials(广东省量子工程与量子材料重点实验室);School of Electronic Science and Engineering (School of Microelectronics), South China Normal University(华南师范大学电子科学与工程学院(微电子学院));Shenzhen University(深圳大学);Columbia University(哥伦比亚大学)
Heterogeneous Element-Aware Cross-Version Differencing of Scientific Documents via Layout-Aware Alignment and Structure-Aware Reasoning
Beijing Renhe Information Technology Co., Ltd.(北京人和信息技术有限公司);Key Laboratory of Digital Publishing and Total Process Management of Scientific and Technical Journals(科技期刊数字出版与全程管理重点实验室)
RAGA: Relation-aware Graph Attention Networks for Global Entity Alignment
School of Software and Microelectronics, Peking University(北京大学软件与微电子学院);National Engineering Research Center for Software Engineering, Peking University(北京大学国家软件工程研究中心);Key Laboratory of High Confidence Software Technologies (PKU), Ministry of Education(教育部高可信软件技术重点实验室(北京大学))
Instituto de Telecomunicações(电信研究所);Unbabel(Unbabel公司);INESC-ID(葡萄牙系统与计算机工程研究所(里斯本分部));Instituto Superior Técnico(高等技术学院);LUMLIS (Lisbon ELLIS Unit)(里斯本ELLIS单元LUMLIS)
最近提出了几种基于神经的度量来评估机器翻译质量。然而,所有这些方法都采用点估计,这在分部一级提供的信息有限。更糟糕的是,它们是在嘈杂、有偏见和缺乏人类判断的情况下训练的,往往导致不可靠的质量预测。在本文中,我们介绍了不确定性感知MT评估和分析的可信度的预测质量。我们将COMET框架与两种不确定性估计方法(Monte Carlo dropout和deep ensembles)相结合,以获得质量分数和置信区间。我们比较了QT21数据集和WMT20指标任务中多个语言对的不确定性感知MT评估方法的性能,并使用MQM注释进行了增强。我们实验不同数量的参考文献,并进一步讨论了有用的不确定性意识的质量估计(没有参考文献),标志可能是关键的翻译错误。
EAMA : Entity-Aware Multimodal Alignment Based Approach for News Image Captioning
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);School of Electronics Engineering and Computer Science, Peking University(北京大学电子工程与计算机科学学院)