RAG / 检索增强生成

Enhancing Retrieval-Augmented Audio Captioning with Generation-Assisted Multimodal Querying and Progressive Learning

Choi Changin, Lim Sungjun, Rhee Wonjong

机构 * Interdisciplinary Program in Artificial Intelligence（人工智能交叉学科项目） ； Department of Intelligence and Information（智能与信息系） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；分类 cs.AI

A Smart Multimodal Healthcare Copilot with Powerful LLM Reasoning

Xuejiao Zhao, Siyan Liu, Su-Yin Yang, Chunyan Miao

机构 * Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY), NTU（联合NTU-UBC老龄化积极生活卓越研究中心（LILY），NTU） ； College of Computing and Data Science, Nanyang Technological University (NTU), Singapore（计算与数据科学学院，南洋理工大学（NTU），新加坡） ； Tan Tock Seng Hospital, Singapore（坦 tok sing 医院，新加坡） ； Woodlands Health, Singapore（伍德兰兹健康，新加坡） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；分类 cs.AI

RADAR: Enhancing Radiology Report Generation with Supplementary Knowledge Injection

Wenjun Hou, Yi Cheng, Kaishuai Xu, Heng Li, Yan Hu, Wenjie Li, Jiang Liu

机构 * Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系） ； Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学可信自主系统研究院和计算机科学与工程系） ； School of Computer Science, University of Nottingham Ningbo China（宁波大学计算机学院） 纠错

专题命中 多模态RAG ：knowledge retrieval(abstract)；分类 cs.CL

Comments Accepted to ACL 2025 main

Tevatron 2.0: Unified Document Retrieval Toolkit across Scale, Language, and Modality

Xueguang Ma, Luyu Gao, Shengyao Zhuang, Jiaqi Samantha Zhan, Jamie Callan, Jimmy Lin

专题命中 多模态RAG ：retriever(abstract)；分类 cs.IR

Comments Accepted in SIGIR 2025 (Demo)

Detecting and Understanding Hateful Contents in Memes Through Captioning and Visual Question-Answering

Ali Anaissi, Junaid Akram, Kunal Chaturvedi, Ali Braytee

机构 * The University of Sydney, School of Computer Science（悉尼大学计算机科学学院） ； University of Technology Sydney, School of Computer Science（新南威尔士大学技术学院） ； University of Technology Sydney, TD School（新南威尔士大学TD学院） ； Australian Catholic University, Peter Faber Business School（澳大利亚天主教大学彼得·法伯商学院） 纠错

专题命中 多模态RAG ：RAG(abstract)；分类 cs.AI

Comments 13 pages, 2 figures, 2025 International Conference on Computational Science

Beyond Text: Implementing Multimodal Large Language Model-Powered Multi-Agent Systems Using a No-Code Platform

Cheonsu Jeong

专题命中 多模态RAG ：RAG(abstract)；分类 cs.AI

Comments 22 pages, 27 figures

Journal ref 2025 Journal of Intelligence and Information Systems

Rotation Invariance in Floor Plan Digitization using Zernike Moments

Marius Graumann, Jan Marius Stürmer, Tobias Koch

专题命中 多模态RAG ：RAG(abstract)；分类 cs.AI

Comments 17 pages, 5 figures

AI-based association analysis for medical imaging using latent-space geometric confounder correction

Xianjing Liu, Bo Li, Meike W. Vernooij, Eppo B. Wolvius, Gennady V. Roshchupkin, Esther E. Bron

专题命中 多模态RAG ：vector search(abstract)；分类 cs.AI

Comments Accepted by Medical Image Analysis

Diffusion Lens: Interpreting Text Encoders in Text-to-Image Pipelines

Michael Toker, Hadas Orgad, Mor Ventura, Dana Arad, Yonatan Belinkov

专题命中 多模态RAG ：knowledge retrieval(abstract)；分类 cs.CL

Comments Published in: ACL 2024 Project webpage: tokeron.github.io/DiffusionLensWeb

OmniQuery: Contextually Augmenting Captured Multimodal Memory to Enable Personal Question Answering

Jiahao Nick Li, Zhuohao Jerry Zhang, Jiaju Ma

专题命中 多模态RAG ：RAG(abstract)；分类 cs.AI

Comments Paper accepted to the 2025 CHI Conference on Human Factors in Computing Systems (CHI 2025)

Multimodal semantic retrieval for product search

Dong Liu, Esther Lopez Ramos

专题命中 多模态RAG ：dense retrieval(abstract)；分类 cs.IR

Comments Accepted at EReL@MIR WWW 2025

Eliza: A Web3 friendly AI Agent Operating System

Shaw Walters, Sam Gao, Shakker Nerd, Feng Da, Warren Williams, Ting-Chien Meng, Amie Chow, Hunter Han, Frank He, Allen Zhang, Ming Wu, Timothy Shen, Maxwell Hu, Jerry Yan

专题命中 多模态RAG ：RAG(abstract)；分类 cs.AI

Comments 20 pages, 5 figures

Improving Multimodal LLMs Ability In Geometry Problem Solving, Reasoning, And Multistep Scoring

Avinash Anand, Raj Jaiswal, Abhishek Dharmadhikari, Atharva Marathe, Harsh Parimal Popat, Harshil Mital, Kritarth Prasad, Rajiv Ratn Shah, Roger Zimmermann

专题命中 多模态RAG ：RAG(abstract)；分类 cs.AI

Comments 15 pages

ViMGuard: A Novel Multi-Modal System for Video Misinformation Guarding

Andrew Kan, Christopher Kan, Zaid Nabulsi

专题命中 多模态RAG ：retrieval augmented generation(abstract)；分类 cs.CL

Comments 7 pages, 2 figures

GeoCoder: Solving Geometry Problems by Generating Modular Code through Vision-Language Models

Aditya Sharma, Aman Dalmia, Mehran Kazemi, Amal Zouaq, Christopher J. Pal

专题命中 多模态RAG ：RAG(abstract)；分类 cs.CL

WorldGPT: Empowering LLM as Multimodal World Model

Zhiqi Ge, Hongzhe Huang, Mingze Zhou, Juncheng Li, Guoming Wang, Siliang Tang, Yueting Zhuang

专题命中 多模态RAG ：knowledge retrieval(abstract)；分类 cs.AI

Comments update v2

Beyond Relevance: Improving User Engagement by Personalization for Short-Video Search

Wentian Bao, Hu Liu, Kai Zheng, Chao Zhang, Shunyu Zhang, Enyun Yu, Wenwu Ou, Yang Song

专题命中 多模态RAG ：dense retrieval(abstract)；分类 cs.IR

Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles

Qiujing Lu, Xuanhan Wang, Yiwei Jiang, Guangming Zhao, Mingyue Ma, Shuo Feng

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；分类 cs.AI

Snap and Diagnose: An Advanced Multimodal Retrieval System for Identifying Plant Diseases in the Wild

Tianqi Wei, Zhi Chen, Xin Yu

专题命中 多模态RAG ：retriever(abstract)；分类 cs.IR

Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos

Changan Chen, Puyuan Peng, Ami Baid, Zihui Xue, Wei-Ning Hsu, David Harwath, Kristen Grauman

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；分类 cs.AI

Comments Project page: https://vision.cs.utexas.edu/projects/action2sound. ECCV 2024 camera-ready version

BSPA: Exploring Black-box Stealthy Prompt Attacks against Image Generators

Yu Tian, Xiao Yang, Yinpeng Dong, Heming Yang, Hang Su, Jun Zhu

专题命中 多模态RAG ：retriever(abstract)；分类 cs.CL

DeepMerge: Deep-Learning-Based Region-Merging for Image Segmentation

Xianwei Lv, Claudio Persello, Wangbin Li, Xiao Huang, Dongping Ming, Alfred Stein

专题命中 多模态RAG ：RAG(abstract)；分类 cs.AI

Video and Audio are Images: A Cross-Modal Mixer for Original Data on Video-Audio Retrieval

Zichen Yuan, Qi Shen, Bingyi Zheng, Yuting Liu, Linying Jiang, Guibing Guo

专题命中 多模态RAG ：retriever(abstract)；分类 cs.IR

PromptCap: Prompt-Guided Task-Aware Image Captioning

Yushi Hu, Hang Hua, Zhengyuan Yang, Weijia Shi, Noah A Smith, Jiebo Luo

专题命中 多模态RAG ：knowledge retrieval(abstract)；分类 cs.CL

Comments Accepted to ICCV 2023

Building Multimodal AI Chatbots

Min Young Lee

专题命中 多模态RAG ：retriever(abstract)；分类 cs.CL

Comments Bachelor's thesis

Semi-Parametric Video-Grounded Text Generation

Sungdong Kim, Jin-Hwa Kim, Jiyoung Lee, Minjoon Seo

专题命中 多模态RAG ：retriever(abstract)；分类 cs.CL

Comments Preprint (16 pages, 5 figures)

COFAR: Commonsense and Factual Reasoning in Image Search

Prajwal Gatti, Abhirama Subramanyam Penamakuri, Revant Teotia, Anand Mishra, Shubhashis Sengupta, Roshni Ramnani

专题命中 多模态RAG ：knowledge retrieval(abstract)；分类 cs.CL

Comments Accepted in AACL-IJCNLP 2022

VILT: Video Instructions Linking for Complex Tasks

Sophie Fischer, Carlos Gemmell, Iain Mackie, Jeffrey Dalton

专题命中 多模态RAG ：dense retrieval(abstract)；分类 cs.IR

Comments 7 pages, IMuR Preprint

Similar Scenes arouse Similar Emotions: Parallel Data Augmentation for Stylized Image Captioning

Guodun Li, Yuchen Zhai, Zehao Lin, Yin Zhang

专题命中 多模态RAG ：retriever(abstract)；分类 cs.CL

Comments Accepted at ACM Multimedia (ACMMM) 2021

VisualSparta: An Embarrassingly Simple Approach to Large-scale Text-to-Image Search with Weighted Bag-of-words

Xiaopeng Lu, Tiancheng Zhao, Kyusong Lee

专题命中 多模态RAG ：vector search(abstract)；分类 cs.CL

Comments Accepted to ACL2021 (10 pages)