RAG / 检索增强生成

CUE-M: Contextual Understanding and Enhanced Search with Multimodal Large Language Model

Dongyoung Go, Taesun Whang, Chanhee Lee, Hwa-Yeon Kim, Sunghoon Park, Seunghwan Ji, Jinho Kim, Dongchan Kim, Young-Bum Kim

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL

Comments Preprint. Under review

RAD: Retrieval-Augmented Decision-Making of Meta-Actions with Vision-Language Models in Autonomous Driving

Yujin Wang, Quanfeng Liu, Zhengxin Jiang, Tianyi Wang, Junfeng Jiao, Hongqing Chu, Bingzhao Gao, Hong Chen

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

On-Board Vision-Language Models for Personalized Autonomous Vehicle Motion Control: System Design and Real-World Validation

Can Cui, Zichong Yang, Yupeng Zhou, Juntong Peng, Sung-Yeon Park, Cong Zhang, Yunsheng Ma, Xu Cao, Wenqian Ye, Yiheng Feng, Jitesh Panchal, Lingxi Li, Yaobin Chen, Ziran Wang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments arXiv admin note: substantial text overlap with arXiv:2410.15281

Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generation

Liwen Sun, James Zhao, Megan Han, Chenyan Xiong

专题命中 多模态RAG ：RAG(abstract)；retriever(abstract)；分类 cs.CL

Comments NAACL 2025 main

Characterizing Network Structure of Anti-Trans Actors on TikTok

Maxyn Leitner, Rebecca Dorn, Fred Morstatter, Kristina Lerman

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments 11 pages, 4 figures. 2 tables

ImageRef-VL: Enabling Contextual Image Referencing in Vision-Language Models

Jingwei Yi, Junhao Yin, Ju Xu, Peng Bao, Yongliang Wang, Wei Fan, Hao Wang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documents

Jun Chen, Dannong Xu, Junjie Fei, Chun-Mei Feng, Mohamed Elhoseiny

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments the correct arxiv version

MMDS: A Multimodal Medical Diagnosis System Integrating Image Analysis and Knowledge-based Departmental Consultation

Yi Ren, HanZhi Zhang, Weibin Li, Jun Fu, Diandong Liu, Tianyi Zhang, Jie He, Licheng Jiao

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

ML-Promise: A Multilingual Dataset for Corporate Promise Verification

Yohei Seki, Hakusen Shu, Anaïs Lhuissier, Hanwool Lee, Juyeon Kang, Min-Yuh Day, Chung-Chi Chen

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL

Comments 6 pages

G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models

Pengyue Jia, Yiding Liu, Xiaopeng Li, Yuhao Wang, Yantong Du, Xiao Han, Xuetao Wei, Shuaiqiang Wang, Dawei Yin, Xiangyu Zhao

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments Accepted to NeurIPS2024

OpenResearcher: Unleashing AI for Accelerated Scientific Research

Yuxiang Zheng, Shichao Sun, Lin Qiu, Dongyu Ru, Cheng Jiayang, Xuefeng Li, Jifan Lin, Binjie Wang, Yun Luo, Renjie Pan, Yang Xu, Qingkai Min, Zizhao Zhang, Yiwen Wang, Wenjie Li, Pengfei Liu

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR

Comments Accepted to Demo track of EMNLP 2024

ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Text

Han Yu, Peikun Guo, Akane Sano

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

IPL: Leveraging Multimodal Large Language Models for Intelligent Product Listing

Kang Chen, Qingheng Zhang, Chengbao Lian, Yixin Ji, Xuwei Liu, Shuguang Han, Guoqiang Wu, Fei Huang, Jufeng Chen

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL

Fortify Your Foundations: Practical Privacy and Security for Foundation Model Deployments In The Cloud

Marcin Chrapek, Anjo Vahldiek-Oberwagner, Marcin Spoczynski, Scott Constable, Mona Vij, Torsten Hoefler

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

OrthoDoc: Multimodal Large Language Model for Assisting Diagnosis in Computed Tomography

Youzhu Jin, Yichen Zhang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments 8 pages, 1 figure

A Learnable Agent Collaboration Network Framework for Personalized Multimodal AI Search Engine

Yunxiao Shi, Min Xu, Haimin Zhang, Xing Zi, Qiang Wu

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR

Comments ACMMM 2024 MMGR WORKSHOP

SearchLVLMs: A Plug-and-Play Framework for Augmenting Large Vision-Language Models by Searching Up-to-Date Internet Knowledge

Chuanhao Li, Zhen Li, Chenchen Jing, Shuo Liu, Wenqi Shao, Yuwei Wu, Ping Luo, Yu Qiao, Kaipeng Zhang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments 13 pages, 6 figures, a plug-and-play framework to augment large vision-language models with up-to-date internet knowledge

LLM4DESIGN: An Automated Multi-Modal System for Architectural and Environmental Design

Ran Chen, Xueqi Yao, Xuhui Jiang

专题命中 多模态RAG ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Towards Unlocking Insights from Logbooks Using AI

Antonin Sulc, Alex Bien, Annika Eichler, Daniel Ratner, Florian Rehm, Frank Mayet, Gregor Hartmann, Hayden Hoschouer, Henrik Tuennermann, Jan Kaiser, Jason St. John, Jennefer Maldonado, Kyle Hazelwood, Raimund Kammering, Thorsten Hellert, Tim Wilksen, Verena Kain, Wan-Lin Hu

专题命中 多模态RAG ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL

Comments 5 pages, 1 figure, 15th International Particle Accelerator Conference

OmniVIC: A Self-Improving Variable Impedance Controller with Vision-Language In-Context Learning for Safe Robotic Manipulation

Heng Zhang, Wei-Hsing Huang, Gokhan Solak, Arash Ajoudani

机构 * Human-Robot Interfaces and Interaction Lab, Istituto Italiano di Tecnologia, Genoa, Italy（人机交互实验室，意大利理工学院，热那亚） ； Georgia Institute of Technology, Atlanta, USA（佐治亚理工学院，美国亚特兰大） 纠错

专题命中 多模态RAG ：RAG(abstract,comments)；retrieval-augmented generation(abstract)

Comments Code, video and RAG dataset are available at \url{https://sites.google.com/view/omni-vic}

UEmbed: Unified Sparse and Dense Multimodal Embeddings

UEmbed：统一稀疏与稠密多模态嵌入

Tingyu Song, Mingxin Li, Yanzhao Zhang, Dingkun Long, Pengjun Xie, Zhijie Nie, Yilun Zhao, Shu Wu

机构 * CASIA（中国科学院自动化研究所） ； Alibaba Group（阿里巴巴集团） ； University of Chinese Academy of Sciences（中国科学院大学） ； Yale University（耶鲁大学） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；分类 cs.IR、cs.CL、cs.AI

AI总结 UEmbed是一种仅解码器的多模态嵌入模型，可单次前向传播生成稀疏与稠密表示，在MMEB-v2和BEIR上表现优异，统一了两类嵌入并支持多模态智能体应用。

PRIMA: Pre-Training with Risk-Integrated Image--Metadata Alignment for Medical Diagnosis with LLM-Based Feature Aggregation

PRIMA：通过大语言模型进行风险集成图像-元数据对齐的医学诊断预训练

Yiqing Wang, Chunming He, Ziyun Yang, Maria Woodward, Ming-Chen Lu, Mercy Pawar, Leslie Niziol, Sina Farsiu

机构 * Department of Biomedical Engineering, Duke University（杜克大学生物医学工程系） ； Department of Ophthalmology and Visual Sciences, University of Michigan（密歇根大学眼科与视觉科学系） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)

AI总结 提出PRIMA框架，通过检索增强生成策展风险-疾病关联专家语料库优化文本编码器，用双编码器预训练策略及四个互补损失函数弥合模态差距，融合特征用于疾病分类，性能优于其他方法。

Superpixel-Based QUBO for Scalable Quantum-Enhanced Medical Image Segmentation

基于超像素的二次无约束二进制优化用于可扩展量子增强医学图像分割

Mohammad Chalhoub, Mahdi Chehimi, Laia Domingo, Omar Alhussein, Ahmed Farouk, Saif Al-Kuwari

机构 * American University of Beirut（贝鲁特美国大学） ； Centre de Visió per Computador (CVC)（计算机视觉中心 (CVC)） ； Ingenii Inc.（英格尼公司） ； Hamad Bin Khalifa University（哈马德·本·哈利法大学） 纠错

专题命中 多模态RAG ：RAG(abstract,abstract_cn)

AI总结 研究针对QUBO用于医学图像分割时的可扩展性挑战，提出基于超像素的QUBO框架，用简单线性迭代聚类分组像素，在区域邻接图上公式化分割。实验表明该方法提升了分割质量、计算速度并减小问题规模，还符合量子退火器连接限制。

Comments 7 pages, 2 figures

OmniRetriever: Any-to-Any Audio-Video-Text Retrieval via Fusion-as-Teacher Distillation

OmniRetriever: 通过融合作为教师蒸馏实现任意到任意的音频-视频-文本检索

Yunze Liu, Chi-Hao Wu, Enmin Zhou, Junxiao Shen

机构 * Memories.ai Research（Memories.ai研究院） 纠错

专题命中 多模态RAG ：RAG(abstract,abstract_cn)

AI总结 提出融合作为教师蒸馏方法，利用三元组嵌入的融合信号训练单模态嵌入，并构建OmniRetriever-7B模型，在零样本检索基准上超越现有方法，同时发布OmniRetriever-Bench基准。

Comments https://yunzeliu.github.io/OmniRetriever/

Decision-Making with Lightweight Confidence-Aware Language Model for Autonomous Driving

基于轻量级置信感知语言模型的自动驾驶决策

Ruoyu Yao, Ruiguo Zhong, Pei Liu, Mingxing Peng, Rui Yang, Jun Ma

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） 纠错

专题命中 多模态RAG ：retrieval augmented generation(abstract)；RAG(abstract)

AI总结 提出一种利用轻量级置信感知语言模型的决策框架，通过多智能体协作生成置信注释的决策演示并蒸馏到双头轻量模型，在nuPlan上实现SOTA成功率和低延迟。

Comments 8 Pages, 3 figures, ITSC 2026

HOG-Layout: Hierarchical 3D Scene Generation, Optimization and Editing via Vision-Language Models

HOG-Layout：通过视觉-语言模型实现层次化3D场景生成、优化与编辑

Haiyan Jiang, Deyu Zhang, Dongdong Weng, Weitao Song, Henry Been-Lirn Duh

机构 * The Hong Kong Polytechnic University（香港理工大学） ； Beijing Institute of Technology（北京理工大学） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)

AI总结 HOG-Layout利用大语言模型和视觉-语言模型实现文本驱动的层次化3D场景生成、优化与实时编辑，通过检索增强生成技术提升语义一致性，结合优化模块增强物理一致性，实现高效场景编辑。

Comments CVPR 2026

Towards Effective Long Video Understanding of Multimodal Large Language Models via One-shot Clip Retrieval

通过一次剪辑检索增强多模态大语言模型的长视频理解

Tao Chen, Shaobo Ju, Qiong Wu, Chenxin Fang, Kun Zhang, Jun Peng, Hui Li, Yiyi Zhou, Rongrong Ji

机构 * Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China（多媒体可信感知与高效计算教育部重点实验室） ； Xiamen University（厦门大学） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)

AI总结 本文提出OneClip-RAG方法，通过一次剪辑检索提升多模态大语言模型对长视频的理解能力，改进知识完整性和语义连贯性，并设计新数据集和训练方案。

VQ-Jarvis: Retrieval-Augmented Video Restoration Agent with Sharp Vision and Fast Thought

VQ-Jarvis：具备锐利视觉与快速思考的检索增强视频修复代理

Xuanyu Zhang, Weiqi Li, Qunliang Xing, Jingfen Xie, Bin Chen, Junlin Li, Li Zhang, Jian Zhang, Shijie Zhao

机构 * School of Electronic and Computer Engineering, Peking University, Shenzhen, China（北京大学电子与计算机工程学院，深圳，中国） ； ByteDance Inc., Shenzhen, China（字节跳动公司，深圳，中国） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)

AI总结 本文提出VQ-Jarvis，一种具备锐利视觉与快速思考的视频修复代理，通过构建大规模视频配对增强数据集和分层操作调度策略，提升视频修复的准确性和效率。

Comments Video restoration, Agent-based restoration

FinMMDocR: Benchmarking Financial Multimodal Reasoning with Scenario Awareness, Document Understanding, and Multi-Step Computation

FinMMDocR：基于场景意识、文档理解与多步骤计算的金融多模态推理基准测试

Zichen Tang, Haihong E, Rongjin Li, Jiacheng Liu, Linwei Jia, Zhuodi Hao, Zhongjun Yang, Yuanze Li, Haolin Tian, Xinyi Hu, Peizhi Zhao, Yuan Liu, Zhengyu Wang, Xianghe Wang, Yiling Huang, Xueyuan Lin, Ruofei Bai, Zijian Xie, Qian Huang, Ruining Cao, Haocheng Gao

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)

AI总结 FinMMDocR通过引入场景意识、文档理解和多步骤计算，推动多模态大语言模型在现实金融场景中的推理能力提升，其包含12种隐含金融场景、9类丰富文档及平均11步推理任务。

Comments Accepted by AAAI-26 Main Track

Journal ref Proc. AAAI 2026 (Vol. 40, No. 30), pages 25858-25866

PaAgent: Portrait-Aware Image Restoration Agent via Subjective-Objective Reinforcement Learning

PaAgent：通过主观-客观强化学习实现的面向人物图像修复代理

Yijian Wang, Qingsen Yan, Jiantao Zhou, Duwei Dai, Wei Dong

机构 * School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机学院） ； Shenzhen Research Institute of Northwestern Polytechnical University（西北工业大学深圳研究院） ； State Key Laboratory of Internet of Things for Smart City, University of Macau（澳门大学智慧城市物联网国家重点实验室） ； National-Local Joint Engineering Research Center of Biodiagnosis and Biotherapy, the Second Affiliated Hospital of Xi’an Jiaotong University（西安交通大学生物诊断与生物治疗国家地方联合工程研究中心） ； College of Information and Control Engineering, Xi’an University of Architecture and Technology（西安建筑科技大学信息与控制工程学院） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)

AI总结 PaAgent通过结合自进化的人物银行和检索增强生成技术，提升图像修复任务中对复杂场景的感知能力，通过主观-客观强化学习策略优化修复工具选择，实验验证其在多种修复基准上的优越性。