Searching for small objects in large images is a task that is both challenging for current deep learning systems and important in numerous real-world applications, such as remote sensing and medical imaging. Thorough scanning of very large images is computationally expensive, particularly at resolutions sufficient to capture small objects. The smaller an object of interest, the more likely it is to be obscured by clutter or otherwise deemed insignificant. We examine these issues in the context of two complementary problems: closed-set object detection and open-set target search. First, we present a method for predicting pixel-level objectness from a low resolution gist image, which we then use to select regions for performing object detection locally at high resolution. This approach has the benefit of not being fixed to a predetermined grid, thereby requiring fewer costly high-resolution glimpses than existing methods. Second, we propose a novel strategy for open-set visual search that seeks to find all instances of a target class which may be previously unseen and is defined by a single image. We interpret both detection problems through a probabilistic, Bayesian lens, whereby the objectness maps produced by our method serve as priors in a maximum-a-posteriori approach to the detection step. We evaluate the end-to-end performance of both the combination of our patch selection strategy with this target search approach and the combination of our patch selection strategy with standard object detection methods. Both elements of our approach are seen to significantly outperform baseline strategies.
Shop The Look: Building a Large Scale Visual Shopping System at Pinterest
Pinterest, Inc.(Pinterest公司)
随着在线内容变得越来越视觉化,通过视觉查询进行搜索的需求相应地增长。Shop The Look是Pinterest的在线购物发现服务,利用视觉搜索使用户能够在图像中找到和购买产品。在这项工作中,我们提供了一个整体的观点,我们如何建立商店的外观,面向购物的视觉搜索系统,以及从解决购物需求的经验教训。我们讨论的主题包括跨目标检测和视觉嵌入的核心技术,实时推理的服务基础设施,以及用于训练/评估数据汇集和人类评估的数据标记方法。我们的系统设计选择的面向用户的影响是通过离线评估,人类相关性判断和在线A/B实验来衡量的。集体改进相当于在端到端人类相关性判断方面累积相对收益超过160%,在参与度方面累积相对收益超过80%。Shop The Look已部署在Pinterest的生产中。
VisualCheXbert: Addressing the Discrepancy Between Radiology Report Labels and Image Labels
Stanford University(斯坦福大学);VinBrain
从自由文本放射学报告中自动提取医学状况对于监督计算机视觉模型解释医学图像至关重要。在这项工作中,我们表明,放射科医生标记报告显着不同意放射科医生标记相应的胸部X射线图像,这降低了质量的报告标签作为图像标签的代理。我们开发和评估的方法,以产生标签的放射学报告,有更好的协议与放射科医生标记图像。我们的最佳性能方法称为VisualCheXbert,它使用生物医学预训练的BERT模型直接从放射学报告映射到图像标签,并通过训练用于从胸部X射线图像中检测医疗状况的计算机视觉模型确定监督信号。我们发现,VisualCheXbert优于使用现有放射学报告标签的方法,平均F1得分为0.14(95%CI 0.12,0.17)。我们还发现,VisualCheXpert更好地同意放射科医生标记胸部X射线图像比放射科医生标记相应的放射学报告之间的平均F1分数在0.12(95% CI 0.09,0.15)和0.21(95% CI 0.18,0.24)之间的几种医疗条件。
Visual-Invariance-Augmented Feature Optimal Alignment for Transferable Adversarial Attacks against Closed-Source MLLMs
Nanyang Technological University(南洋理工大学);Northeastern University(东北大学);Chongqing University(重庆大学);University of Chinese Academy of Sciences(中国科学院大学);Sun Yat-sen University(中山大学)
MSCI: Addressing CLIP's Inherent Limitations for Compositional Zero-Shot Learning
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Key Laboratory of Social Computing and Cognitive Intelligence (Dalian University of Technology), Ministry of Education, China(社会科学计算与认知智能重点实验室(大连理工大学));The Hong Kong Polytechnic University(香港理工大学)
Ask and Remember: A Questions-Only Replay Strategy for Continual Visual Question Answering
LTCI, Télécom-Paris, Institut Polytechnique de Paris(LTCI, Télécom-Paris, Institut Polytechnique de Paris);Inria, LJK, Univ. Grenoble Alpes(Inria, LJK, 火车头 Grenoble Alpes);Universitat Autónoma de Barcelona(Autonomous University of Barcelona)
CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Update
School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);State Key Laboratory of General Artificial Intelligence, BIGAI(北京智源人工智能研究院通用人工智能国家重点实验室);Beijing Jiaotong University(北京交通大学);Department of Automation, Tsinghua University(清华大学自动化系)
P$^2$-DPO: Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization
Guangdong Provincial Key Laboratory of Computational AI Models and Cognitive Intelligence, School of Computer Science & Engineering, South China University of Technology(广东省计算人工智能模型与认知智能重点实验室,计算机科学与工程学院,华南理工大学);Pazhou Lab, Guangzhou, China(琶洲实验室,广州,中国);Engineering Research Center of the Ministry of Education on Health Intelligent Perception and Paralleled Digital-Human, Guangzhou, China(教育部健康智能感知与并行数字人工程研究中心,广州,中国)
Enhancing Glass Defect Detection with Diffusion Models: Addressing Imbalanced Datasets in Manufacturing Quality Control
Department of Applied Statistics & Operations Research (ASOR), Bowling Green State University, Bowling Green, OH, USA(应用统计与运筹学系,布罗金格林州立大学);Data Science Department, Actual Reality Technologies, OH, USA(数据科学系,Actual Reality Technologies)
HART: High-Resolution Annotation-Free Reasoning Technique through a Closed-loop Framework
State Key Laboratory of Novel Software Technology(新型软件技术国家重点实验室);Institute of Brain-inspired Intelligence(脑启发式智能研究院);Shenzhen Research Institute of Nanjing University(南京大学深圳研究院)
Surgical-LVLM: Learning to Adapt Large Vision-Language Model for Grounded Visual Question Answering in Robotic Surgery
The Chinese University of Hong Kong(香港中文大学);Universiti Malaya(马来亚大学);Centre for Artificial Intelligence and Robotics, HKISI-CAS(香港智能系统研究所人工智能与机器人中心);University College London(伦敦大学学院)