Existing manifold learning methods are not appropriate for image retrieval task, because most of them are unable to process query image and they have much additional computational cost especially for large scale database. Therefore, we propose the iterative manifold embedding (IME) layer, of which the weights are learned off-line by unsupervised strategy, to explore the intrinsic manifolds by incomplete data. On the large scale database that contains 27000 images, IME layer is more than 120 times faster than other manifold learning methods to embed the original representations at query time. We embed the original descriptors of database images which lie on manifold in a high dimensional space into manifold-based representations iteratively to generate the IME representations in off-line learning stage. According to the original descriptors and the IME representations of database images, we estimate the weights of IME layer by ridge regression. In on-line retrieval stage, we employ the IME layer to map the original representation of query image with ignorable time cost (2 milliseconds). We experiment on five public standard datasets for image retrieval. The proposed IME layer significantly outperforms related dimension reduction methods and manifold learning methods. Without post-processing, Our IME layer achieves a boost in performance of state-of-the-art image retrieval methods with post-processing on most datasets, and needs less computational cost.
Multi-granularity Association Learning Framework for on-the-fly Fine-Grained Sketch-based Image Retrieval
Chongqing University of Posts and Telecommunications（重庆邮电大学）；University of California, Riverside（加利福尼亚大学河滨分校）
基于细粒度草图的图像检索（FG-SBIR）解决了在给定的查询草图中检索特定照片的问题。但是，它的广泛适用性受到限制，因为大多数人很难画出完整的草图，而且绘制过程往往需要时间。在这项研究中，我们的目标是用尽可能少的笔画数（不完整的草图）检索目标照片，命名为即时FG-SBIR（Bhunia et al. 2020），它在绘画开始时就开始检索每个笔画。我们认为，在每张照片的草图绘制片段中，这些不完整的草图之间存在着显著的相关性。为了更有效地学习照片和不完整草图之间共享的联合嵌入空间，我们提出了一个多粒度关联学习框架，进一步优化了所有不完整草图的嵌入空间。具体来说，基于草图的完整性，我们可以将一个完整的草图集分为几个阶段，每个阶段对应一个简单的线性映射层。此外，该框架引导当前草图的向量空间表示逼近后续草图的向量空间表示，实现了笔划较少草图的检索性能逼近笔划较多草图的检索性能。在实验中，我们提出了更现实的挑战，我们的方法在两个公开的细粒度草图检索数据集上实现了优于最先进的方法和替代基线的早期检索效率。
IMRAM: Iterative Matching with Recurrent Attention Memory for Cross-Modal Image-Text Retrieval
Tsinghua University（清华大学）；Kwai Ads Platform（快手广告平台）；Microsoft Research（微软研究院）；Kwai Seattle AI Lab（快手西雅图人工智能实验室）；Kwai FeDA Lab（快手联邦学习实验室）；Kwai AI Platform（快手人工智能平台）；University of Warwick（华威大学）
Learning a Text-Video Embedding from Incomplete and Heterogeneous Data
École Normale Supérieure（巴黎高等师范学院）；CNRS（法国国家科学研究中心）；PSL Research University（巴黎文理研究大学）；Czech Institute of Informatics, Robotics and Cybernetics at the Czech Technical University in Prague（捷克布拉格技术大学捷克信息学、机器人学与控制论研究所）；Inria（法国国家信息与自动化研究所）；CIIRC（捷克信息学、机器人学与控制论研究所）
MERLIN: Multimodal Embedding Refinement via LLM-based Iterative Navigation for Text-Video Retrieval-Rerank Pipeline
Buzzni AI Lab（Buzzni人工智能实验室）；KAIST（韩国科学技术院）；Mathpresso Inc.（Mathpresso公司）；UC Berkeley（加州大学伯克利分校）；Fainders AI（Fainders人工智能公司）；Seoul National University（首尔大学）
I2I-PR: Deep Iterative Refinement for Phase Retrieval using Image-to-Image Diffusion Models
Department of Electrical Engineering, Middle East Technical University (METU)（电子工程系，中东技术大学）；Department of Applied Mathematics and Computer Science, Technical University of Denmark（应用数学与计算机科学系，丹麦技术大学）
Retrieving to Recover: Towards Incomplete Audio-Visual Question Answering via Semantic-consistent Purification
School of Computing and Information Technology, Great Bay University（大湾大学计算机与信息学院）；Dongguan Key Laboratory for Intelligence and Information Technology（东莞智能与信息科技重点实验室）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；College of Computer Science, Nankai University（南开大学计算机学院）；School of mathematics, Sun Yat-sen University（中山大学数学学院）
An iterative energy-based multimodal transformer for joint retrieval of wheat soil moisture, leaf area index, and plant height from Sentinel-1 and Sentinel-2 time series
Department of Urban and Environmental Policy and Program, Tufts University（Tufts大学城市与环境政策与项目系）；Electrical and Computer Engineering Department, Mississippi State University（密苏里州立大学电气与计算机工程系）；Department of Physics, Indian Institute of Technology (BHU)（印度理工学院（BHU）物理系）；Institute of Environment and Sustainable Development, Banaras Hindu University（巴纳尔斯赫尔大学环境与可持续发展研究所）
Chain of Evidence: Pixel-Level Visual Attribution for Iterative Retrieval-Augmented Generation
National Engineering Research Center for Software Engineering, Peking University（软件工程国家级工程研究中心，北京大学）；City University of Hong Kong（香港城市大学）；Peking University（北京大学）；Tencent Technology（腾讯科技）
提出Chain of Evidence (CoE)框架，利用视觉语言模型直接对检索到的文档截图进行推理，输出精确边界框以可视化完整推理链，解决迭代检索增强生成中的粗粒度归因和视觉语义丢失问题。
