视觉大模型 / VLM - arXivDaily 专题

2601.13995 2026-01-21 cs.CL 50%

From Tags to Trees: Structuring Fine-Grained Knowledge for Controllable Data Selection in LLM Instruction Tuning

从标签到树：为LLM指令微调中的可控数据选择构建细粒度知识

Zihan Niu, Wenping Hu, Junmin Chen, Xiyue Wang, Tong Xu, Ruiming Tang

机构 * University of Science and Technology of China（中国科学技术大学）； Klear Team, Kuaishou Technology（快手科技Klear团队）

专题命中 VLM训练与架构：grounding(abstract)

AI总结 TAGS通过构建细粒度知识树，实现LLM指令微调中可控数据选择的高效采样与知识对齐。

2601.13384 2026-01-21 cs.SE cs.CL 50%

From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuning

从补全到编辑：通过搜索和替换指令微调解锁上下文感知的代码填充

Jiajun Zhang, Zeyu Cui, Jiaxi Yang, Lei Zhang, Yuheng Jing, Zeyao Ma, Tianyi Bai, Zilei Wang, Qiang Liu, Liang Wang, Binyuan Hui, Junyang Lin

机构 * USTC（University of Science and Technology of China）； Alibaba Group（阿里巴巴集团）； CASIA（Chinese Academy of Sciences Institute of Automation）； SIAT（Institute of Automation, Chinese Academy of Sciences）

专题命中 VLM训练与架构：grounding(abstract)

AI总结通过搜索和替换指令微调，SRI框架实现了上下文感知的代码填充，提升了补全性能并保持低延迟。

2601.03648 2026-01-21 cs.CL 50%

ELO: Efficient Layer-Specific Optimization for Continual Pretraining of Multilingual LLMs

ELO：面向多语言大语言模型持续预训练的高效分层优化

HanGyeol Yoo, ChangSu Choi, Minjun Kim, Seohyun Song, SeungWoo Song, Inho Won, Jongyoul Park, Cheoneum Park, KyungTae Lim

机构 * Seoul National University of Science and Technology（首尔科学技术大学）； LG CNS ； Korea Advanced Institute of Science and Technology（韩国科学技术院）； Hanbat National University（汉 bat 国立大学）

专题命中 VLM训练与架构：MLLM(abstract)

AI总结 ELO通过分层优化提升多语言大模型持续预训练效率，实现6.46倍加速和6.2%性能提升。

Comments 12 pages, Accepted to EACL 2026 (Industrial Track)

2601.07143 2026-01-13 cs.HC 50%

EZBlender: Efficient 3D Editing with Plan-and-ReAct Agent

EZBlender：基于计划与反应代理的高效3D编辑

Hao Wang, Wenhui Zhu, Shao Tang, Zhipeng Wang, Xuanzhao Dong, Xin Li, Xiwen Chen, Ashish Bastola, Xinhao Huang, Yalin Wang, Abolfazl Razi

专题命中 VLM训练与架构：VLM(abstract)

AI总结 EZBlender通过结合计划与反应机制，实现高效且语义忠实的3D编辑，提升Human AI协作效率与经济性。

2408.12800 2026-01-13 cs.MM 50%

Cap2Sum: Learning to Summarize Videos by Generating Captions

Cap2Sum: 通过生成描述来学习视频摘要

Cairong Zhao, Chutian Wang, Zifan Song, Guosheng Hu, Haonan Chen, Xiaofan Zhai

专题命中 VLM训练与架构：vision-language model(abstract)

AI总结 Cap2Sum通过生成视频描述来学习视频摘要，利用密集视频描述注释提升模型性能和泛化能力。

Comments 13 pages, 4 figures

2601.06798 2026-01-13 cs.IR 50%

Unleashing the Native Recommendation Potential: LLM-Based Generative Recommendation via Structured Term Identifiers

释放原生推荐潜力：基于结构化术语标识符的LLM生成推荐

Zhiyang Zhang, Junda She, Kuo Cai, Bo Chen, Shiyao Wang, Xinchen Luo, Qiang Luo, Ruiming Tang, Han Li, Kun Gai, Guorui Zhou

专题命中 VLM训练与架构：grounding(abstract)

AI总结本文提出GRLM框架，通过结构化术语标识符提升生成推荐系统的性能和通用性。

2601.04736 2026-01-09 cs.CL 50%

AM$^3$Safety: Towards Data Efficient Alignment of Multi-modal Multi-turn Safety for MLLMs

AM$^3$Safety: 向多模态多轮安全对齐的数据高效方法

Han Zhu, Jiale Chen, Chengkun Cai, Shengjie Sun, Haoran Li, Yujin Zhou, Chi-Min Chan, Pengcheng Wen, Lei Li, Sirui Han, Yike Guo

机构 * Hong Kong University of Science and Technology（香港科技大学）； Zhongshan School of Medicine, SUN YAT-SEN UNIVERSITY（中山医学院，孙中山大学）； University of Edinburgh（爱丁堡大学）； University of Washington（华盛顿大学）

专题命中 VLM训练与架构：LLaVA(abstract)

AI总结 AM$^3$Safety通过结合冷启动拒绝阶段和组相对策略优化，有效提升多模态多轮对话的安全性，降低攻击成功率并增强模型的无害与帮助维度。

2506.21170 2026-01-06 cs.CL 50%

Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modeling

Cosmos: 用于文本扩散建模的压缩和平滑潜在空间

Viacheslav Meshchaninov, Egor Chimbulatov, Alexander Shabalin, Aleksandr Abramov, Dmitry Vetrov

机构 * HSE University（俄罗斯高等经济大学）； Constructor University（建设大学）； SaluteDevices

专题命中 VLM训练与架构：grounding(abstract)

AI总结 Cosmos通过压缩和平滑的潜在空间提升文本生成效率，实现比传统方法更快的推理速度和相当的生成质量。

2510.25138 2026-01-01 cs.RO 50%

Learning Spatial-Aware Manipulation Ordering

学习空间感知的操作顺序

Yuxiang Yan, Zhiyuan Zhou, Xin Gao, Guanghao Li, Shenglin Li, Jiaqi Chen, Qunyan Pu, Jian Pu

机构 * Fudan University（复旦大学）； Shanghai YinCheng Intelligent CO., LTD（上海英成智能有限公司）； Stanford University（斯坦福大学）

专题命中 VLM训练与架构：vision-language model(abstract)

AI总结 OrderMind通过空间感知学习在杂乱环境中实现高效鲁棒操作的统一框架。

Comments Accepted to NeurIPS 2025

2512.22741 2025-12-30 cs.CL 50%

Text-Routed Sparse Mixture-of-Experts Model with Explanation and Temporal Alignment for Multi-Modal Sentiment Analysis

基于解释和时序对齐的文本引导稀疏专家混合模型用于多模态情感分析

Dongning Rao, Yunbiao Zeng, Zhihua Jiang, Jujian Lv

专题命中 VLM训练与架构：MLLM(abstract)

AI总结本文提出基于解释和时序对齐的文本引导稀疏专家混合模型，用于提升多模态情感分析的性能。

Comments 9 pages, 9 figures, accepted by AAAI 2026

2512.18117 2025-12-23 cs.IR 50%

Factorized Transport Alignment for Multimodal and Multiview E-commerce Representation Learning

因子化运输对多模态和多视角电商表示学习

Xiwen Chen, Yen-Chieh Lien, Susan Liu, María Castaños, Abolfazl Razi, Xiaoting Zhao, Congzhe Su

专题命中 VLM训练与架构：vision-language model(abstract)

AI总结本文提出因子化运输框架，通过统一多模态和多视角学习，提升电商场景下的检索性能。

Comments Accepted by WSDM'26

2505.14972 2025-12-23 cs.CL 50%

Multimodal Cultural Safety: Evaluation Framework and Alignment Strategies

多模态文化安全：评估框架与对齐策略

Haoyi Qiu, Kung-Hsiang Huang, Ruichen Zheng, Jiao Sun, Nanyun Peng

机构 * University of California, Los Angeles（加州大学洛杉矶分校）； Salesforce AI Research（Salesforce人工智能研究）； Google DeepMind（谷歌DeepMind）

专题命中 VLM训练与架构：vision-language model(abstract)

AI总结本文提出CROSS基准和CROSS-Eval框架，评估多模态模型的文化安全能力，发现提升推理能力可改善文化对齐，但需结合监督微调和偏好微调策略以增强文化合规性。

2502.11361 2025-12-23 cs.CL 50%

VLDBench Evaluating Multimodal Disinformation with Regulatory Alignment

VLDBench：评估具有监管对齐的多模态虚假信息

Shaina Raza, Ashmal Vayani, Aditya Jain, Aravind Narayanan, Vahid Reza Khazaie, Syed Raza Bashir, Elham Dolatabadi, Gias Uddin, Christos Emmanouilidis, Rizwan Qureshi, Mubarak Shah

专题命中 VLM训练与架构：vision-language model(abstract)

AI总结 VLDBench 是首个多模态虚假信息检测基准，通过大规模标注数据提升检测准确率，支持 AI 管治框架下的可信虚假信息分析。

Comments Accepted in Information Fusion Journal

2512.17154 2025-12-22 cs.SD 50%

InstructDubber: Instruction-based Alignment for Zero-shot Movie Dubbing

InstructDubber：基于指令的零样本电影配音对齐

Zhedong Zhang, Liang Li, Gaoxiang Cong, Chunshan Liu, Yuhan Gao, Xiaowan Wang, Tao Gu, Yuankai Qi

机构 * VIPL group, ICT, CAS（中国科学院信息科技研究所VIPL小组）

专题命中 VLM训练与架构：multimodal large language model(abstract)

AI总结 InstructDubber通过指令生成和持续时间蒸馏模块，实现鲁棒的领域内和零样本电影配音对齐。

Comments Accepted by AAAI2026

2505.01146 2025-12-19 q-bio.OT 50%

Retrieval-Augmented Generation in Biomedicine: A Survey of Technologies, Datasets, and Clinical Applications

生物医学中的检索增强生成：技术、数据集和临床应用的综述

Jiawei He, Boya Zhang, Hossein Rouhizadeh, Yingjian Chen, Rui Yang, Jin Lu, Xudong Chen, Nan Liu, Douglas Teodoro

专题命中 VLM训练与架构：grounding(abstract)

AI总结本文综述了生物医学中检索增强生成技术的发展，探讨了其在临床应用中的挑战与未来发展方向。

Comments 49 pages

2508.00522 2025-12-16 cs.CL 50%

Efficiently Seeking Flat Minima for Better Generalization in Fine-Tuning Large Language Models and Beyond

高效寻找平坦极小值以提升大型语言模型及其他模型的泛化能力

Jiaxin Deng, Qingcheng Zhu, Junbiao Pang, Linlin Yang, Zhongqian Fu, Baochang Zhang

专题命中 VLM训练与架构：vision-language model(abstract)

AI总结本文提出EFMLoRA，通过寻找LoRA的平坦极小值提升模型泛化能力，实验表明其在效率和性能上均优于传统LoRA方法。

2512.11755 2025-12-15 cs.CL 50%

SUMFORU: An LLM-Based Review Summarization Framework for Personalized Purchase Decision Support

SUMFORU: 一种基于LLM的个性化购买决策支持的评论摘要框架

Yuming Feng, Xinrui Jiang

专题命中 VLM训练与架构：grounding(abstract)

AI总结 SUMFORU通过可调节的多元对齐方法，实现个性化购买决策支持，结合高质量数据管道和两阶段对齐流程，提升摘要的一致性、基础性和偏好对齐性能。

Comments Code available at https://github.com/Harry20030331/SumForU

2512.10741 2025-12-12 cs.CL 50%

TRIDENT: A Redundant Architecture for Caribbean-Accented Emergency Speech Triage

TRIDENT：一种用于加勒比口音紧急语音分诊的冗余架构

Elroy Galbraith, Chadwick Sutherland, Donahue Morgan

专题命中 VLM训练与架构：grounding(abstract)

AI总结 TRIDENT通过结合加勒比口音优化的ASR、实体提取和生物声学检测，为调度员提供三种信号以提高紧急分诊效率，解决非标准英语变体识别问题。

2502.02603 2025-12-11 eess.AS cs.CL cs.SD 50%

SEAL: Speech Embedding Alignment Learning for Speech Large Language Model with Retrieval-Augmented Generation

SEAL：用于带有检索增强生成的语音大语言模型的语音嵌入对齐学习

Chunyu Sun, Bingyu Liu, Zhichao Cui, Junhan Shi, Anbin Qi, Tian-hao Zhang, Dinghao Zhou, Lewei Lu

机构 * SenseTime Research（商汤科技研究院）

专题命中 VLM训练与架构：multimodal large language model(abstract)

AI总结 SEAL通过统一的语音和文本嵌入框架，提升语音大语言模型的检索效率与准确性，减少延迟并增强多模态检索能力。

2512.06022 2025-12-09 cs.SD cs.MM 50%

DreamFoley: Scalable VLMs for High-Fidelity Video-to-Audio Generation

DreamFoley: 用于高保真视频到音频生成的可扩展视觉-语言模型

Fu Li, Weichao Zhao, You Li, Zhichao Zhou, Dongliang He

机构 * Bytedance Intelligent Creation Lab（字节跳动智能创作实验室）； Zhejiang University（浙江大学）

专题命中 VLM训练与架构：vision-language model(abstract)

AI总结 DreamFoley通过结合视觉-语言模型，提出了一种可扩展的视频到音频生成方法，实现了高保真音频生成并优化了训练效率与质量的平衡。

Comments 10 pages; Bytedance

2512.03976 2025-12-04 cs.CL 50%

Adapting Large Language Models to Low-Resource Tibetan: A Two-Stage Continual and Supervised Fine-Tuning Study

将大型语言模型适应于低资源藏语：一种两阶段持续和监督微调研究

Lifeng Chen, Ryan Lai, Tianming Liu

专题命中 VLM训练与架构：grounding(abstract)

AI总结本研究通过两阶段方法将Qwen2.5-3B适应到藏语，通过持续预训练和监督微调提升翻译质量，实现低资源语言的模型适应。

2505.10792 2025-12-04 cs.CL 50%

Finetune-RAG: Fine-Tuning Language Models to Resist Hallucination in Retrieval-Augmented Generation

Finetune-RAG: 通过微调语言模型抵抗检索增强生成中的幻觉

Zhan Peng Lee, Andre Lin, Calvin Tan

机构 * Pints AI Labs（Pints AI 实验室）

专题命中 VLM训练与架构：grounding(abstract)

AI总结 Finetune-RAG通过微调方法提升LLM事实准确性，提出首个RAG训练数据集和压力测试评估流程。

2511.22576 2025-12-01 cs.MM 50%

A Progressive Evaluation Framework for Multicultural Analysis of Story Visualization

一种用于故事可视化多元文化分析的渐进评估框架

Janak Kapuriya, Ali Hatami, Paul Buitelaar

专题命中 VLM训练与架构：MLLM(abstract)

AI总结本文提出一种渐进多元文化评估框架，通过五个新指标评估故事可视化模型在不同文化下的表现，揭示模型在文化适当性和视觉美学上的差异。

2511.20683 2025-11-27 cs.CL 50%

Dynamic Template Selection for Output Token Generation Optimization: MLP-Based and Transformer Approaches

动态模板选择用于输出标记生成优化：基于MLP和Transformer的方法

Bharadwaj Yadavalli

专题命中 VLM训练与架构：grounding(abstract)

AI总结本文提出动态模板选择方法，通过MLP和Transformer路由策略优化输出标记生成，实现成本降低与响应质量的平衡。

Comments 20 pages, 4 figures, includes production-scale experiments across OpenAI GPT-4, Google Gemini, and Anthropic Claude; code available upon request

2511.18709 2025-11-25 cs.RO 50%

Autonomous Surface Selection For Manipulator-Based UV Disinfection In Hospitals Using Foundation Models

基于基础模型的医院机器人紫外线消毒表面自主选择

Xueyan Oh, Jonathan Her, Zhixiang Ong, Brandon Koh, Yun Hann Tan, U-Xuan Tan

机构 * Engineering Product Development Pillar, Singapore University of Technology and Design, Singapore（新加坡科技设计大学工程产品开发部门）； Centre for Healthcare Assistive & Robotics Technology, Singapore（新加坡医疗辅助与机器人技术中心）； National Centre for Infectious Diseases, Singapore（新加坡传染病国家中心）

专题命中 VLM训练与架构：VLM(abstract)

AI总结本研究提出基于基础模型的机器人紫外线消毒表面自主选择方法，通过视觉语言模型辅助分割细化，提高消毒效率并减少误判，实验证明其在实际应用中的有效性。

Comments 7 pages, 7 figures; This paper has been accepted by IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

2511.11035 2025-11-17 cs.MA 50%

GraphMASAL: A Graph-based Multi-Agent System for Adaptive Learning

Biqing Zeng, Mengquan Liu, Zongwei Zhen

专题命中 VLM训练与架构：grounding(abstract)

Comments 9 pages, 3 figures,submitted to AAMAS 2026

2509.06031 2025-11-17 cs.RO 50%

GELATO: Multi-Instruction Trajectory Reshaping via Geometry-Aware Multiagent-based Orchestration

Junhui Huang, Yuhe Gong, Changsheng Li, Xingguang Duan, Luis Figueredo

机构 * Beijing Institute of Technology (BIT)（北京理工大学）； School of Computer Science, University of Nottingham（诺丁汉大学计算机学院）

专题命中 VLM训练与架构：VLM(abstract)

2505.16774 2025-11-13 cs.CL 50%

IFEval-Audio: Benchmarking Instruction-Following Capability in Audio-based Large Language Models

Yiming Gao, Bin Wang, Chengwei Wei, Shuo Sun, AiTi Aw

机构 * Nanyang Technological University (NTU)（南洋理工大学）； MiroMind（米罗Mind）； Institute for Infocomm Research (I 2 R)（信息与通信研究院）； A*STAR（科技研究局）

专题命中 VLM训练与架构：vision-language model(abstract)

Comments Link: https://github.com/AudioLLMs/AudioBench/tree/main/IFEval-Audio

2511.08498 2025-11-12 astro-ph.IM astro-ph.SR 50%

A Simple Ray Acceleration Structure for Non-LTE Radiative Transfer

Christopher M. J. Osborne

专题命中 VLM训练与架构：VLM(abstract)

Comments 12 pages, 7 figures. Accepted for publication in RASTI

2510.27569 2025-11-03 cs.CL 50%

MARAG-R1: Beyond Single Retriever via Reinforcement-Learned Multi-Tool Agentic Retrieval

Qi Luo, Xiaonan Li, Yuxin Wang, Tingshuo Fan, Yuan Li, Xinchi Chen, Xipeng Qiu

机构 * School of Computer Science, Fudan University（复旦大学计算机学院）

专题命中 VLM训练与架构：grounding(abstract)

1. VLM训练与架构 5058 篇

From Tags to Trees: Structuring Fine-Grained Knowledge for Controllable Data Selection in LLM Instruction Tuning

From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuning

ELO: Efficient Layer-Specific Optimization for Continual Pretraining of Multilingual LLMs

EZBlender: Efficient 3D Editing with Plan-and-ReAct Agent

Cap2Sum: Learning to Summarize Videos by Generating Captions

Unleashing the Native Recommendation Potential: LLM-Based Generative Recommendation via Structured Term Identifiers

AM$^3$Safety: Towards Data Efficient Alignment of Multi-modal Multi-turn Safety for MLLMs

Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modeling

Learning Spatial-Aware Manipulation Ordering

Text-Routed Sparse Mixture-of-Experts Model with Explanation and Temporal Alignment for Multi-Modal Sentiment Analysis

Factorized Transport Alignment for Multimodal and Multiview E-commerce Representation Learning

Multimodal Cultural Safety: Evaluation Framework and Alignment Strategies

VLDBench Evaluating Multimodal Disinformation with Regulatory Alignment

InstructDubber: Instruction-based Alignment for Zero-shot Movie Dubbing

Retrieval-Augmented Generation in Biomedicine: A Survey of Technologies, Datasets, and Clinical Applications

Efficiently Seeking Flat Minima for Better Generalization in Fine-Tuning Large Language Models and Beyond

SUMFORU: An LLM-Based Review Summarization Framework for Personalized Purchase Decision Support

TRIDENT: A Redundant Architecture for Caribbean-Accented Emergency Speech Triage

SEAL: Speech Embedding Alignment Learning for Speech Large Language Model with Retrieval-Augmented Generation

DreamFoley: Scalable VLMs for High-Fidelity Video-to-Audio Generation

Adapting Large Language Models to Low-Resource Tibetan: A Two-Stage Continual and Supervised Fine-Tuning Study

Finetune-RAG: Fine-Tuning Language Models to Resist Hallucination in Retrieval-Augmented Generation

A Progressive Evaluation Framework for Multicultural Analysis of Story Visualization

Dynamic Template Selection for Output Token Generation Optimization: MLP-Based and Transformer Approaches

Autonomous Surface Selection For Manipulator-Based UV Disinfection In Hospitals Using Foundation Models

GraphMASAL: A Graph-based Multi-Agent System for Adaptive Learning

GELATO: Multi-Instruction Trajectory Reshaping via Geometry-Aware Multiagent-based Orchestration

IFEval-Audio: Benchmarking Instruction-Following Capability in Audio-based Large Language Models

A Simple Ray Acceleration Structure for Non-LTE Radiative Transfer

MARAG-R1: Beyond Single Retriever via Reinforcement-Learned Multi-Tool Agentic Retrieval