DEFT: Distribution-guided Efficient Fine-Tuning for Human Alignment
Southern University of Science and Technology(南方科技大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Science and Technology of China(中国科学技术大学);University of Copenhagen(哥本哈根大学)
Liang Zhu, Feiteng Fang, Yuelin Bai, Longze Chen, Zhexiang Zhang, Minghuan Tan, Min Yang
Reinforcement Learning from Human Feedback (RLHF), using algorithms like Proximal Policy Optimization (PPO), aligns Large Language Models (LLMs) with human values but is costly and unstable. Alternatives have been proposed to replace PPO or integrate Supervised Fine-Tuning (SFT) and contrastive learning for direct fine-tuning and value alignment. However, these methods still require voluminous data to learn preferences and may weaken the generalization ability of LLMs. To further enhance alignment efficiency and performance while mitigating the loss of generalization ability, this paper introduces Distribution-guided Efficient Fine-Tuning (DEFT), an efficient alignment framework incorporating data filtering and distributional guidance by calculating the differential distribution reward based on the output distribution of language model and the discrepancy distribution of preference data. A small yet high-quality subset is filtered from the raw data using a differential distribution reward, which is then incorporated into existing alignment methods to guide the model's output distribution. Experimental results demonstrate that the methods enhanced by DEFT outperform the original methods in both alignment capability and generalization ability, with significantly reduced training time.
DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference
Westlake University(西湖大学);Zhejiang University(浙江大学);Carnegie Mellon University(卡内基梅隆大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Hong Kong University of Science and Technology(香港科技大学)
DeFT提出了一种基于Flash树注意力的硬件高效解码算法,通过KV-Guided Grouping和Flattened Tree KV Splitting优化树结构LLM推理中的KV缓存分区与负载均衡,显著减少IO冗余并提升GPU利用率,实现最高2.23倍端到端加速。
EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment
DAMO Academy, Alibaba Group(阿里巴巴达摩院);The First Affiliated Hospital of Zhejiang Chinese Medical University(浙江中医药大学附属第一医院);Shanghai Jiao Tong University(上海交通大学);Hupan Lab(湖畔实验室);Zhejiang University(浙江大学)
Beyond representational alignment with brain-guided language models for robust reasoning
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University(北京大学通用人工智能国家重点实验室、智能科学与技术学院);Department of Psychological and Cognitive Sciences, Tsinghua University(清华大学心理与认知科学系);Microsoft Research Asia(微软亚洲研究院)
Eye-gaze Guided Multi-modal Alignment for Medical Representation Learning
School of Automation, Northwestern Polytechnical University(西北工业大学自动化学院);City University of Hong Kong(香港城市大学);The University of Texas at Arlington(德克萨斯大学阿灵顿分校);School of Biomedical Engineering, ShanghaiTech University(上海科技大学生物医学工程学院);Shanghai United Imaging Intelligence Co., Ltd.(上海联影智能医疗科技有限公司);Shanghai Clinical Research and Trial Center(上海临床研究与试验中心);University of Georgia(佐治亚大学);Massachusetts General Hospital(麻省总医院)
Self-Guided Adaptive Safety Alignment: Synthesizing and Internalizing Guidelines in Reasoning Models
Beijing Key Lab of Traffic Data Analysis and Mining(北京交通大数据分析与挖掘重点实验室);Beijing Jiaotong University(北京交通大学);School of Information Technology and Management(信息科技与管理学院);University of International Business and Economics(国际经济贸易大学)
UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models
University of California San Diego(加利福尼亚大学圣迭戈分校);University of Southern California(南加利福尼亚大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Carnegie Mellon University(卡内基梅隆大学)