RAG / 检索增强生成

SK-VQA: Synthetic Knowledge Generation at Scale for Training Context-Augmented Multimodal LLMs

Xin Su, Man Luo, Kris W Pan, Tien Pei Chou, Vasudev Lal, Phillip Howard

机构 * Intel Labs（英特尔实验室） ； Amazon（亚马逊） 纠错

专题命中 多模态RAG ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL

Comments ICML 2025 Spotlight Oral

SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models

Jiawei Zhang, Xuan Yang, Taiqi Wang, Yu Yao, Aleksandr Petiushko, Bo Li

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

ImageRAG: Enhancing Ultra High Resolution Remote Sensing Imagery Analysis with ImageRAG

Zilun Zhang, Haozhan Shen, Tiancheng Zhao, Zian Guan, Bin Chen, Yuhao Wang, Xu Jia, Yuxiang Cai, Yongheng Shang, Jianwei Yin

机构 * College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院） ； Binjiang Research Institute of Zhejiang University（浙江大学滨江研究院） ； School of Software Engineering of Zhejiang University（浙江大学软件工程学院） ； Polytechnic Institute of Zhejiang University（浙江大学 polytechnic 院） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments Accepted by IEEE Geoscience and Remote Sensing Magazine

Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models

Yifan Jia, Kailin Jiang, Yuyang Liang, Qihan Ren, Yi Xin, Rui Yang, Fenze Feng, Mingcai Chen, Hengyang Lu, Haozhe Wang, Xiaoye Qu, Dongrui Liu, Lizhen Cui, Yuntao Du

机构 * Joint SDU-NTU Centre for Artificial Intelligence Research&School of Software（山东大学与南京工业大学人工智能研究中心及软件学院） ； University of Science and Technology of China（中国科学技术大学） ； Shanghai Jiaotong University（上海交通大学） ； Nanjing University（南京大学） ； Nanjing University of Posts and Telecommunications（南京邮电大学） ； Jiangnan University（江南大学） ； The Hong Kong University of Science and Technology（香港科学与技术大学） ； Shanghai AI Laboratory（上海人工智能实验室） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments The source code is available at https://github.com/MLLMKCBENCH/MLLMKC

A Multimodal Multi-Agent Framework for Radiology Report Generation

Ziruo Yi, Ting Xiao, Mark V. Albert

机构 * University of North Texas（北卡罗来纳州立大学） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Facilitating Video Story Interaction with Multi-Agent Collaborative System

Yiwen Zhang, Jianing Hao, Zhan Wang, Hongling Sheng, Wei Zeng

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments Prepared and submitted in 2024

Retrieval-augmented in-context learning for multimodal large language models in disease classification

Zaifu Zhan, Shuang Zhou, Xiaoshan Zhou, Yongkang Xiao, Jun Wang, Jiawen Deng, He Zhu, Yu Hou, Rui Zhang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments 17 Pages, 1 figure, 7 tables

MRAMG-Bench: A Comprehensive Benchmark for Advancing Multimodal Retrieval-Augmented Multimodal Generation

Qinhan Yu, Zhiyou Xiao, Binghui Li, Zhengren Wang, Chong Chen, Wentao Zhang

机构 * Peking University（北京大学） ； Huawei Cloud BU（华为云业务单元） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR

Comments Published as a conference paper at SIGIR 2025; 11 pages

On the Feasibility of Using MultiModal LLMs to Execute AR Social Engineering Attacks

Ting Bi, Chenghang Ye, Zheyu Yang, Ziyi Zhou, Cui Tang, Jun Zhang, Zui Tao, Kailong Wang, Liting Zhou, Yang Yang, Tianlong Yu

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Reasoning LLMs for User-Aware Multimodal Conversational Agents

Hamed Rahimi, Jeanne Cattoni, Meriem Beghili, Mouad Abrini, Mahdi Khoramshahi, Maribel Pino, Mohamed Chetouani

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Agentic Multimodal AI for Hyperpersonalized B2B and B2C Advertising in Competitive Markets: An AI-Driven Competitive Advertising Framework

Sakhinana Sagar Srinivas, Akash Das, Shivam Gupta, Venkataramana Runkana

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

CUE-M: Contextual Understanding and Enhanced Search with Multimodal Large Language Model

Dongyoung Go, Taesun Whang, Chanhee Lee, Hwa-Yeon Kim, Sunghoon Park, Seunghwan Ji, Jinho Kim, Dongchan Kim, Young-Bum Kim

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL

Comments Preprint. Under review

RAD: Retrieval-Augmented Decision-Making of Meta-Actions with Vision-Language Models in Autonomous Driving

Yujin Wang, Quanfeng Liu, Zhengxin Jiang, Tianyi Wang, Junfeng Jiao, Hongqing Chu, Bingzhao Gao, Hong Chen

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

On-Board Vision-Language Models for Personalized Autonomous Vehicle Motion Control: System Design and Real-World Validation

Can Cui, Zichong Yang, Yupeng Zhou, Juntong Peng, Sung-Yeon Park, Cong Zhang, Yunsheng Ma, Xu Cao, Wenqian Ye, Yiheng Feng, Jitesh Panchal, Lingxi Li, Yaobin Chen, Ziran Wang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments arXiv admin note: substantial text overlap with arXiv:2410.15281

Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generation

Liwen Sun, James Zhao, Megan Han, Chenyan Xiong

专题命中 多模态RAG ：RAG(abstract)；retriever(abstract)；分类 cs.CL

Comments NAACL 2025 main

Characterizing Network Structure of Anti-Trans Actors on TikTok

Maxyn Leitner, Rebecca Dorn, Fred Morstatter, Kristina Lerman

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments 11 pages, 4 figures. 2 tables

ImageRef-VL: Enabling Contextual Image Referencing in Vision-Language Models

Jingwei Yi, Junhao Yin, Ju Xu, Peng Bao, Yongliang Wang, Wei Fan, Hao Wang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documents

Jun Chen, Dannong Xu, Junjie Fei, Chun-Mei Feng, Mohamed Elhoseiny

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments the correct arxiv version

MMDS: A Multimodal Medical Diagnosis System Integrating Image Analysis and Knowledge-based Departmental Consultation

Yi Ren, HanZhi Zhang, Weibin Li, Jun Fu, Diandong Liu, Tianyi Zhang, Jie He, Licheng Jiao

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

ML-Promise: A Multilingual Dataset for Corporate Promise Verification

Yohei Seki, Hakusen Shu, Anaïs Lhuissier, Hanwool Lee, Juyeon Kang, Min-Yuh Day, Chung-Chi Chen

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL

Comments 6 pages

G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models

Pengyue Jia, Yiding Liu, Xiaopeng Li, Yuhao Wang, Yantong Du, Xiao Han, Xuetao Wei, Shuaiqiang Wang, Dawei Yin, Xiangyu Zhao

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments Accepted to NeurIPS2024

OpenResearcher: Unleashing AI for Accelerated Scientific Research

Yuxiang Zheng, Shichao Sun, Lin Qiu, Dongyu Ru, Cheng Jiayang, Xuefeng Li, Jifan Lin, Binjie Wang, Yun Luo, Renjie Pan, Yang Xu, Qingkai Min, Zizhao Zhang, Yiwen Wang, Wenjie Li, Pengfei Liu

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR

Comments Accepted to Demo track of EMNLP 2024

ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Text

Han Yu, Peikun Guo, Akane Sano

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

IPL: Leveraging Multimodal Large Language Models for Intelligent Product Listing

Kang Chen, Qingheng Zhang, Chengbao Lian, Yixin Ji, Xuwei Liu, Shuguang Han, Guoqiang Wu, Fei Huang, Jufeng Chen

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL

Fortify Your Foundations: Practical Privacy and Security for Foundation Model Deployments In The Cloud

Marcin Chrapek, Anjo Vahldiek-Oberwagner, Marcin Spoczynski, Scott Constable, Mona Vij, Torsten Hoefler

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

OrthoDoc: Multimodal Large Language Model for Assisting Diagnosis in Computed Tomography

Youzhu Jin, Yichen Zhang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments 8 pages, 1 figure

A Learnable Agent Collaboration Network Framework for Personalized Multimodal AI Search Engine

Yunxiao Shi, Min Xu, Haimin Zhang, Xing Zi, Qiang Wu

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR

Comments ACMMM 2024 MMGR WORKSHOP

SearchLVLMs: A Plug-and-Play Framework for Augmenting Large Vision-Language Models by Searching Up-to-Date Internet Knowledge

Chuanhao Li, Zhen Li, Chenchen Jing, Shuo Liu, Wenqi Shao, Yuwei Wu, Ping Luo, Yu Qiao, Kaipeng Zhang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments 13 pages, 6 figures, a plug-and-play framework to augment large vision-language models with up-to-date internet knowledge

LLM4DESIGN: An Automated Multi-Modal System for Architectural and Environmental Design

Ran Chen, Xueqi Yao, Xuhui Jiang

专题命中 多模态RAG ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Towards Unlocking Insights from Logbooks Using AI

Antonin Sulc, Alex Bien, Annika Eichler, Daniel Ratner, Florian Rehm, Frank Mayet, Gregor Hartmann, Hayden Hoschouer, Henrik Tuennermann, Jan Kaiser, Jason St. John, Jennefer Maldonado, Kyle Hazelwood, Raimund Kammering, Thorsten Hellert, Tim Wilksen, Verena Kain, Wan-Lin Hu

专题命中 多模态RAG ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL

Comments 5 pages, 1 figure, 15th International Particle Accelerator Conference