SemPA: Improving Sentence Embeddings of Large Language Models through Semantic Preference Alignment
College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机科学与软件工程学院);School of Computer Science and Electronic Engineering, University of Surrey(Surrey大学计算机科学与电子工程学院)
Traditional sentence embedding methods employ token-level contrastive learning on non-generative pre-trained models. Recently, there have emerged embedding methods based on generative large language models (LLMs). These methods either rely on fixed prompt templates or involve modifications to the model architecture. The former lacks further optimization of the model and results in limited performance, while the latter alters the internal computational mechanisms of the model, thereby compromising its generative capabilities. We propose SemPA, a novel approach that boosts the sentence representations while preserving the generative ability of LLMs via semantic preference alignment. We leverage sentence-level Direct Preference Optimization (DPO) to efficiently optimize LLMs on a paraphrase generation task, where the model learns to discriminate semantically equivalent sentences while preserving inherent generative capacity. Theoretically, we establish a formal connection between DPO and contrastive learning under the Plackett-Luce model framework. Empirically, experimental results on both semantic textual similarity tasks and various benchmarks for LLMs show that SemPA achieves better semantic representations without sacrificing the inherent generation capability of LLMs.
ASPO: Adaptive Sentence-Level Preference Optimization for Fine-Grained Multimodal Reasoning
Northwestern Polytechnical University, School of Automation, Xi’an, China(西北工业大学自动化学院,西安,中国);Northwestern Polytechnical University, School of Cybersecurity, Xi’an, China(西北工业大学网络安全学院,西安,中国);Binjiang Institute of Artificial Intelligence, ZJUT, Hangzhou, China(浙大宁波理工学院滨海人工智能研究院,杭州,中国);Alibaba Group, Hangzhou, China(阿里巴巴集团,杭州,中国)
Black Big Boxes: Tracing Adjective Order Preferences in Large Language Models
CLCG(认知语言学研究中心);ILS(语言学研究所);ILLC(语言学研究所);CIMeC(计算机科学与工程学院);University of Groningen(格罗宁根大学);Utrecht University(乌得勒支大学);University of Amsterdam(阿姆斯特丹大学);University of Trento(特伦托大学)
Cross-Preference Learning for Sentence-Level and Context-Aware Machine Translation
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);School of Computer Science and Artificial Intelligence, Zhengzhou University(郑州大学计算机与人工智能学院);Huawei Translation Services Center(华为翻译服务中心)
HPRO: Hierarchical Progressive Reward Optimization via Preference Extraction for Emotional Text-to-Speech
South China University of Technology, China(华南理工大学,中国);Huya Inc., China(欢丫公司,中国);Tongyi Fun Team, Alibaba Group, China(通义 Fun 团队,阿里巴巴集团,中国);Foshan University, China(佛山大学,中国)
DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models
Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence(浙江医学影像人工智能重点实验室);Zhejiang University(浙江大学);SUTD(新加坡科技设计大学);Princeton University(普林斯顿大学);Peking University(北京大学);Nanyang Technological University(南洋理工大学);A*STAR Centre for Frontier AI Research(A*STAR前沿人工智能研究中心)
Human Label Variation as Stable Signal: Learning Annotator-Specific Explanation Behavior via Cross-Annotator Preference Optimization
MaiNLP;Center for Information and Language Processing(信息与语言处理中心);LMU Munich(慕尼黑大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);University of Vienna(维也纳大学);LTL;University of Cambridge(剑桥大学)
Stream Aligner: Efficient Sentence-Level Alignment via Distribution Induction
Institute for AI, Peking University(北京大学人工智能研究院);State Key Laboratory of General Artificial Intelligence, Institute for AI, Peking University(北京大学人工智能研究院通用人工智能国家重点实验室)
Revisiting Recurrent Networks for Paraphrastic Sentence Embeddings
Toyota Technological Institute at Chicago(芝加哥丰田工业大学)
我们考虑了学习通用的、释义的句子嵌入的问题,重温了Wieting et al.(2016 b)的设置。虽然他们发现LSTM循环网络的表现不如单词平均,但我们提出了几个发展,这些发展共同产生了相反的结论。这些包括对句子对而不是短语对进行训练,平均状态以表示序列,以及积极地正则化。这些改进了迁移学习和监督设置中的LSTM。我们还介绍了一个新的经常性架构,门控经常性平均网络,这是平均和LSTM的启发,同时优于他们两个。我们分析我们学习的模型,找到对特定词性和依赖关系的偏好的证据。
How Large Language Models Get Stuck: Early structure with persistent errors
Department of Statistics(统计学系);University of Connecticut, Storrs(康涅狄格大学斯托尔分校);Department of Linguistics(语言学系);Department of Psychological Sciences(心理学科学系)
Quantifier Scope Interpretation in Language Learners and LLMs
Department of English, Purdue University(英语系,普渡大学);Department of Linguistics, Purdue University(语言学系,普渡大学);School of Languages and Cultures, Purdue University(语言与文化学院,普渡大学)
Event knowledge in large language models: the gap between the impossible and the unlikely
Massachusetts Institute of Technology(麻省理工学院);McGovern Institute for Brain Research(麦戈文脑研究所);University of Bologna(博洛尼亚大学);Hong Kong Polytechnic University(香港理工大学);University of Washington(华盛顿大学);University of Pisa(比萨大学)
Underutilization of Syntactic Processing by Chinese Learners of English in Comprehending English Sentences, Evidenced from Adapted Garden-Path Ambiguity Experiment
Token-level Accept or Reject: A Micro Alignment Approach for Large Language Models
Hong Kong Polytechnic University(香港理工大学);MemTensor (Shanghai) Technology Co., Ltd(MemTensor(上海)科技有限公司);University of Science and Technology of China(中国科学技术大学);China Telecom Corporation Limited Beijing Research Institute(中国电信北京研究院);Nanjing University of Information Science and Technology(南京信息工程大学)
ConvoDrift: A Multi-Turn Conversational Dataset for Modeling Stylistic Tone Evolution
Informatics Institute of Technology, Sri Lanka(斯里兰卡信息技术学院);National Health and Medical Research Council, Australia(澳大利亚国家健康与医学研究理事会);Luxembourg Centre for Systems Biomedicine (LCSB), University of Luxembourg(卢森堡大学系统生物医学中心)
Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation
Instituto de Telecomunicações(电信研究所);Unbabel(Unbabel公司);Instituto Superior Técnico, Universidade de Lisboa(里斯本大学高等技术学院);Carnegie Mellon University(卡内基梅隆大学);CentraleSupélec, Université Paris-Saclay(巴黎萨克雷大学中央高等电力学院);ELLIS Unit Lisbon(里斯本ELLIS单元)