We explore multiple instance verification, a problem setting in which a query instance is verified against a bag of target instances with heterogeneous, unknown relevancy. We show that naive adaptations of attention-based multiple instance learning (MIL) methods and standard verification methods like Siamese neural networks are unsuitable for this setting: directly combining state-of-the-art (SOTA) MIL methods and Siamese networks is shown to be no better, and sometimes significantly worse, than a simple baseline model. Postulating that this may be caused by the failure of the representation of the target bag to incorporate the query instance, we introduce a new pooling approach named "cross-attention pooling" (CAP). Under the CAP framework, we propose two novel attention functions to address the challenge of distinguishing between highly similar instances in a target bag. Through empirical studies on three different verification tasks, we demonstrate that CAP outperforms adaptations of SOTA MIL methods and the baseline by substantial margins, in terms of both classification accuracy and the ability to detect key instances. The superior ability to identify key instances is attributed to the new attention functions by ablation studies. We share our code at https://github.com/xxweka/MIV.
Stress-Testing Neural Network Verifiers with Provably Robust Instances
Department of Statistics & Data Science, University of California, Los Angeles(统计与数据科学系,加州大学洛杉矶分校);Department of Mathematics, University of California, Los Angeles(数学系,加州大学洛杉矶分校);Max Planck Institute for Mathematics in the Sciences, Leipzig(科学数学研究所,莱比锡)
SoundnessBench: A Soundness Benchmark for Neural Network Verifiers
University of California, Los Angeles(加州大学洛杉矶分校);Georgia Institute of Technology(佐治亚理工学院);Duke University(杜克大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Riverside(加州大学河滨分校)