Prototypical part neural networks (ProtoPartNNs), namely PROTOPNET and its derivatives, are an intrinsically interpretable approach to machine learning. Their prototype learning scheme enables intuitive explanations of the form, this (prototype) looks like that (testing image patch). But, does this actually look like that? In this work, we delve into why object part localization and associated heat maps in past work are misleading. Rather than localizing to object parts, existing ProtoPartNNs localize to the entire image, contrary to generated explanatory visualizations. We argue that detraction from these underlying issues is due to the alluring nature of visualizations and an over-reliance on intuition. To alleviate these issues, we devise new receptive field-based architectural constraints for meaningful localization and a principled pixel space mapping for ProtoPartNNs. To improve interpretability, we propose additional architectural improvements, including a simplified classification head. We also make additional corrections to PROTOPNET and its derivatives, such as the use of a validation set, rather than a test set, to evaluate generalization during training. Our approach, PIXPNET (Pixel-grounded Prototypical part Network), is the only ProtoPartNN that truly learns and localizes to prototypical object parts. We demonstrate that PIXPNET achieves quantifiably improved interpretability without sacrificing accuracy.
GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensing
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Australian National University(澳大利亚国立大学);The University of Western Australia(西澳大利亚大学)
Seeing through Light and Darkness: Sensor-Physics Grounded Deblurring HDR NeRF from Single-Exposure Images and Events
State Key Laboratory of Virtual Reality Technology and Systems, SCSE & QRI, Beihang University(北京航空航天大学虚拟现实技术与系统国家重点实验室);School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院)
Motion-Grounded Video Reasoning: Understanding and Perceiving Motion at Pixel Level
University of Central Florida(中佛罗里达大学);University of Western Australia(西澳大学);Amazon Web Services(亚马逊网络服务);The University of Texas at Dallas(德克萨斯大学达拉斯分校)
ProtoQuant: Quantization of Prototypical Parts For General and Fine-Grained Image Classification
Jagiellonian University, Faculty of Mathematics and Computer Science(杰洛尼莫夫大学数学与计算机科学系);Jagiellonian University, Doctoral School of Exact and Natural Sciences(杰洛尼莫夫大学精确与自然科学研究博士学院)
PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection
Inria, École normale supérieure, CNRS, PSL Research University(法国国家信息与自动化研究所,巴黎高等师范学院,法国国家科学研究中心,巴黎文理研究大学);Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague(捷克信息学、机器人与控制论研究所,布拉格捷克理工大学)
Mohamed bin Zayed University of AI(穆罕默德·本·扎耶德人工智能大学);Australian National University(澳大利亚国立大学);Aalto University(阿尔托大学);Carnegie Mellon University(卡内基梅隆大学);University of California - Merced(加州大学默塞德分校);Linköping University(林雪平大学);Google Research(谷歌研究院)
GrapeSplat: Geometry-Grounded Reconstruction via Amalgamated Pose-Free Encoding for Feed-Forward 3D Gaussian Splatting
National Taiwan University(国立台湾大学);National Taiwan University of Science and Technology(国立台湾科技大学);NTU AI Center of Research Excellence (NTU AI-CoRE)(台湾大学人工智能卓越研究中心);VinUniversity(Vin大学)
UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoning
The Hong Kong Polytechnic University(香港理工大学);ARC Lab, Tencent PCG(腾讯PCG ARC实验室);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);vivo Mobile Communication Co.(vivo移动通信公司);MindWingman Technology (Shenzhen) Co., Ltd.(深圳MindWingman技术有限公司)
Physics-Informed Untrained Learning for RGB-Guided Superresolution Single-Pixel Hyperspectral Imaging
College of Science, Northwest Agriculture & Forestry University(西北农林科技大学理学院);Key Laboratory of Photoelectronic Imaging Technology and System, School of Optics and Photonics, Beijing Institute of Technology(北京理工大学光电学院光电成像技术与系统教育部重点实验室);State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology(华中科技大学数字制造装备与技术国家重点实验室)