语言大模型 / LLM

Instruction Tuning with and without Context: Behavioral Shifts and Downstream Impact

带有和不带上下文的指令微调：行为变化和下游影响

Hyunji Lee, Seunghyun Yoon, Yunjae Won, Hanseok Oh, Geewook Kim, Trung Bui, Franck Dernoncourt, Elias Stengel-Eskin, Mohit Bansal, Minjoon Seo

机构 * UNC Chapel Hill（北卡罗来纳大学教堂山分校） ； Adobe Research（Adobe研究） ； KAIST AI（韩国科学技术院人工智能实验室） ； Mila – Quebec AI Institute（魁北克人工智能研究所） ； NAVER Cloud AI（NAVER云人工智能） 纠错

专题命中 指令微调 ：instruction tuning(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

AI总结 本文研究了指令微调中带上下文与不带上下文对模型行为和下游性能的影响，发现上下文增强训练能提升模型基础性并减少幻觉，同时提出在实际部署中分离上下文模型以获得更稳健的性能。

Investigating the Multilingual Calibration Effects of Language Model Instruction-Tuning

探究语言模型指令微调的多语言校准效应

Jerry Huang, Peng Lu, Qiuhao Zeng, Yusuke Iwasawa, Yutaka Matsuo, Sarath Chandar, Edison Marrese-Taylor, Irene Li

机构 * Mila - Quebec AI Institute（魁北克人工智能研究所） ； Université de Montréal（蒙特利尔大学） ； The University of Tokyo（东京大学） ； Western University（西方大学） ； Vector Institute（向量研究所） ； Polytechnique Montréal（蒙特利尔理工学院） ； CIFAR AI Chair（CIFAR人工智能主席） ； AIST（日本产业技术综合研究所） 纠错

专题命中 指令微调 ：language model(title,abstract)；large language model(abstract)；foundation model(abstract)；SFT(abstract)

AI总结 本研究探讨了多语言环境下语言模型指令微调对校准的影响，发现高资源语言SFT数据能显著提升模型置信度，但准确性提升有限，揭示了标准SFT在多语言中的局限性。

Comments Accepted to The 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL)

Rethinking Supervised Fine-Tuning: Emphasizing Key Answer Tokens for Improved LLM Accuracy

重新思考监督微调：强调关键答案标记以提高LLM准确性

Xiaofeng Shi, Qian Kou, Yuduo Li, Hua Zhou

机构 * Beijing Academy of Artificial Intelligence (BAAI)（北京人工智能研究院） ； Beijing Jiaotong University (BJTU)（北京交通大学） 纠错

专题命中 指令微调 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

AI总结 SFTKey通过两阶段训练方案，强调关键答案标记以提高LLM在复杂推理任务中的准确性。

T-SHIRT: Token-Selective Hierarchical Data Selection for Instruction Tuning

T-SHIRT: 令牌选择性层次数据选择用于指令微调

Yanjun Fu, Faisal Hamman, Sanghamitra Dutta

机构 * University of Maryland, College Park（马里兰大学学院公园分校） 纠错

专题命中 指令微调 ：instruction tuning(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

AI总结 T-SHIRT通过令牌选择性层次数据选择方法提升指令微调效率，使小规模数据集训练效果优于大规模数据集。

Comments NeurIPS 2025

Deflanderization for Game Dialogue: Balancing Character Authenticity with Task Execution in LLM-based NPCs

Pasin Buakhaw, Kun Kerdthaisong, Phuree Phenhiran, Pitikorn Khlaisamniang, Supasate Vorathammathorn, Piyalitt Ittichaiwong, Nutchanon Yongsatianchot

机构 * Department of Computer Engineering and Digital Technology, Faculty of Engineering, Chulalongkorn University（朱拉隆梭大学工程学院计算机工程与数字技术系） ； Faculty of Engineering, Thammasat School of Engineering, Thammasat University（泰国 Thammasat 大学工程学院） ； Artificial Intelligence Association of Thailand（泰国人工智能协会） ； School of Biomedical Engineering & Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与成像科学学院） ； Siriraj Informatics and Data Innovation Center (SIData+), Faculty of Medicine, Siriraj Hospital, Mahidol University（玛希诺大学医学学院西里拉医院信息与数据创新中心（SIData+）） 纠错

专题命中 指令微调 ：LLM(title)；large language model(abstract)；language model(abstract)；SFT(abstract)

Calibrated Language Models and How to Find Them with Label Smoothing

Jerry Huang, Peng Lu, Qiuhao Zeng

专题命中 指令微调 ：language model(title,abstract)；large language model(abstract)；instruction tuning(abstract)；SFT(abstract)

Comments Accepted to the Forty-second International Conference on Machine Learning (ICML) 2025. First two authors contributed equally. Official proceedings version available at https://proceedings.mlr.press/v267/huang25w.html

DP-Adam-AC: Privacy-preserving Fine-Tuning of Localizable Language Models Using Adam Optimization with Adaptive Clipping

Ruoxing Yang

机构 * Georgetown University（乔治城大学） 纠错

专题命中 指令微调 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；small language model(abstract)

Edge-FIT: Federated Instruction Tuning of Quantized LLMs for Privacy-Preserving Smart Home Environments

Vinay Venkatesh, Vamsidhar R Kamanuru, Lav Kumar, Nikita Kothari

机构 * University of California, San Diego（加州大学圣地亚哥分校） 纠错

专题命中 指令微调 ：instruction tuning(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 7 pages, 1 figure

Efficient Layer-wise LLM Fine-tuning for Revision Intention Prediction

Zhexiong Liu, Diane Litman

机构 * Department of Computer Science, Learning Research & Development Center University of Pittsburgh（计算机科学系、学习研究与开发中心匹兹堡大学） 纠错

专题命中 指令微调 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Comments In The Conference on Empirical Methods in Natural Language Processing (EMNLP), November 2025

Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training

Junlin Han, Shengbang Tong, David Fan, Yufan Ren, Koustuv Sinha, Philip Torr, Filippos Kokkinos

机构 * Meta Superintelligence Labs（Meta 超智能实验室） ； University of Oxford（牛津大学） 纠错

专题命中 指令微调 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Comments Project page: https://junlinhan.github.io/projects/lsbs/

Leveraging What's Overfixed: Post-Correction via LLM Grammatical Error Overcorrection

Taehee Park, Heejin Do, Gary Geunbae Lee

机构 * Graduate School of Artificial Intelligence, POSTECH, South Korea（POSTECH人工智能研究生院） ； ETH Zurich, ETH AI Center（苏黎世联邦理工学院人工智能中心） ； Department of Computer Science and Engineering, POSTECH, South Korea（POSTECH计算机科学与工程系） 纠错

专题命中 指令微调 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；small language model(abstract)

Comments EMNLP 2025

DeepResonance: Enhancing Multimodal Music Understanding via Music-centric Multi-way Instruction Tuning

Zhuoyuan Mao, Mengjie Zhao, Qiyu Wu, Hiromi Wakaki, Yuki Mitsufuji

机构 * Sony Group Corporation（索尼集团公司） ； Sony AI（索尼人工智能） 纠错

专题命中 指令微调 ：instruction tuning(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted to EMNLP 2025 main conference

Conan-Embedding-v2: Training an LLM from Scratch for Text Embeddings

Shiyu Li, Yang Tang, Ruijie Liu, Shi-Zhe Chen, Xi Chen

机构 * Basic Algorithm Center, PCG, Tencent（基础算法中心、PCG、腾讯） 纠错

专题命中 指令微调 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments EMNLP 2025 Oral

CoMMIT: Coordinated Multimodal Instruction Tuning

Xintong Li, Junda Wu, Tong Yu, Yu Wang, Xiang Chen, Jiuxiang Gu, Lina Yao, Julian McAuley, Jingbo Shang

机构 * University of California, San Diego（加州大学圣地亚哥分校） ； Adobe Research（Adobe研究） ； The University of New South Wales（新南威尔士大学） ； CSIRO’s Data61（澳大利亚联邦科学与工业研究组织的数据61） 纠错

专题命中 指令微调 ：instruction tuning(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 9 pages

Generative Foundation Model for Structured and Unstructured Electronic Health Records

Sonish Sivarajkumar, Hang Zhang, Yuelyu Ji, Maneesh Bilalpur, Xizhi Wu, Chenyu Li, Min Gu Kwak, Shyam Visweswaran, Yanshan Wang

专题命中 指令微调 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

MASteer: Multi-Agent Adaptive Steer Strategy for End-to-End LLM Trustworthiness Repair

Changqing Li, Tianlin Li, Xiaohan Zhang, Aishan Liu, Li Pan

专题命中 指令微调 ：LLM(title)；large language model(abstract)；language model(abstract)；SFT(abstract)

Tackling Distribution Shift in LLM via KILO: Knowledge-Instructed Learning for Continual Adaptation

Iing Muttakhiroh, Thomas Fevens

机构 * Concordia University（康科迪亚大学） 纠错

专题命中 指令微调 ：LLM(title)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Enhancing LLM Knowledge Learning through Generalization

Mingkang Zhu, Xi Chen, Zhongdao Wang, Bei Yu, Hengshuang Zhao, Jiaya Jia

专题命中 指令微调 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Compromising Honesty and Harmlessness in Language Models via Deception Attacks

Laurène Vaugrante, Francesca Carlon, Maluna Menke, Thilo Hagendorff

专题命中 指令微调 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；prompting(abstract)

Watermarking Language Models through Language Models

Agnibh Dasgupta, Abdullah Tanvir, Xin Zhong

机构 * Department of Computer Science, University of Nebraska Omaha（内布拉斯加大学奥马哈分校计算机科学系） 纠错

专题命中 指令微调 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；prompting(abstract)

ClimateChat: Designing Data and Methods for Instruction Tuning LLMs to Answer Climate Change Queries

Zhou Chen, Xiao Wang, Yuanhong Liao, Ming Lin, Yuqi Bai

机构 * Tsinghua University（清华大学） 纠错

专题命中 指令微调 ：instruction tuning(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments ICLR 2025 camera ready, 13 pages, 4 figures, 4 tables

LLM Social Simulations Are a Promising Research Method

Jacy Reese Anthis, Ryan Liu, Sean M. Richardson, Austin C. Kozlowski, Bernard Koch, James Evans, Erik Brynjolfsson, Michael Bernstein

专题命中 指令微调 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Published at ICML 2025

EvalYaks: Instruction Tuning Datasets and LoRA Fine-tuned Models for Automated Scoring of CEFR B2 Speaking Assessment Transcripts

Nicy Scaria, Silvester John Joseph Kennedy, Thomas Latinovich, Deepak Subramani

机构 * Computational and Data Science, IISc, Bangalore, India（计算机与数据科学，IISc，班加罗尔，印度） ； Talking Yak, Inc., Cedarburg, Wisconsin, USA（Talking Yak公司， Cedarburg，威斯康星州，美国） ； Talking Yak English Learning Private Limited, Bangalore, India（Talking Yak英语学习私人有限公司，班加罗尔，印度） 纠错

专题命中 指令微调 ：instruction tuning(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Retrieval-Augmented Fine-Tuning With Preference Optimization For Visual Program Generation

Deokhyung Kang, Jeonghun Cho, Yejin Jeon, Sunbin Jang, Minsub Lee, Jawoon Cho, Gary Geunbae Lee

专题命中 指令微调 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Accepted at ACL 2025 (Main, long paper)

FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain

Rohan Deb, Kiran Thekumparampil, Kousha Kalantari, Gaurush Hiranandani, Shoham Sabach, Branislav Kveton

专题命中 指令微调 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；SFT(abstract)

Span-level Emotion-Cause-Category Triplet Extraction with Instruction Tuning LLMs and Data Augmentation

Xiangju Li, Dong Yang, Xiaogang Zhu, Faliang Huang, Peng Zhang, Zhongying Zhao

专题命中 指令微调 ：instruction tuning(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

MergeIT: From Selection to Merging for Efficient Instruction Tuning

Hongyi Cai, Yuqian Fu, Hongming Fu, Bo Zhao

专题命中 指令微调 ：instruction tuning(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuning

Ang Li, Yichuan Mo, Mingjie Li, Yifei Wang, Yisen Wang

专题命中 指令微调 ：prompting(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarking

Benjamin Feuer, Micah Goldblum, Teresa Datta, Sanjana Nambiar, Raz Besaleli, Samuel Dooley, Max Cembalest, John P. Dickerson

专题命中 指令微调 ：LLM(title,abstract)；post-training(abstract)；SFT(abstract)；preference optimization(abstract)

Comments ICLR 2025

Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinement

Maosong Cao, Taolin Zhang, Mo Li, Chuyu Zhang, Yunxin Liu, Haodong Duan, Songyang Zhang, Kai Chen

专题命中 指令微调 ：LLM(title)；large language model(abstract)；language model(abstract)；post-training(abstract)

Comments Tech Report. Github: https://github.com/InternLM/Condor