LILaC：基于分层组件图的开放域多模态多跳检索中的后期交互
LILaC: Late Interacting in Layered Component Graph for Open-domain Multimodal Multihop Retrieval
AI总结：
LILaC通过分层组件图和后期交互子图检索方法，提升开放域多模态多跳检索的精度与效率。
摘要：
多模态文档检索旨在从由文本、表格和视觉元素组成的文档中检索出与查询相关的组件。一个有效的多模态检索器需要处理两个主要挑战：（1）减轻由固定、单一粒度检索单元引起的无关内容的影响；（2）通过有效捕捉文档内和跨文档组件之间的语义关系来支持多跳推理。为了解决这些挑战，我们提出了LILaC，一个具有两个核心创新的多模态检索框架。首先，我们引入了分层组件图，明确地在两个层次上表示多模态信息——每个层次分别代表粗粒度和细粒度——从而实现高效且精确的推理。其次，我们开发了一种基于后期交互的子图检索方法，一种基于边的方法，首先通过粗粒度节点高效生成候选，然后通过后期交互进行细粒度推理。大量实验表明，LILaC在所有五个基准测试中均取得了最先进的检索性能，特别是在没有额外微调的情况下。我们已将相关工具在github.com/joohyung00/lilac上公开发布。
英文摘要：
Multimodal document retrieval aims to retrieve query-relevant components from documents composed of textual, tabular, and visual elements. An effective multimodal retriever needs to handle two main challenges: (1) mitigate the effect of irrelevant contents caused by fixed, single-granular retrieval units, and (2) support multihop reasoning by effectively capturing semantic relationships among components within and across documents. To address these challenges, we propose LILaC, a multimodal retrieval framework featuring two core innovations. First, we introduce a layered component graph, explicitly representing multimodal information at two layers - each representing coarse and fine granularity - facilitating efficient yet precise reasoning. Second, we develop a late-interaction-based subgraph retrieval method, an edge-based approach that initially identifies coarse-grained nodes for efficient candidate generation, then performs fine-grained reasoning via late interaction. Extensive experiments demonstrate that LILaC achieves state-of-the-art retrieval performance on all five benchmarks, notably without additional fine-tuning. We make the artifacts publicly available at github.com/joohyung00/lilac.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
