Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decoding
National University of Singapore（新加坡国立大学）；University of California, Merced（加州大学默塞德分校）；University of California, Los Angeles（加州大学洛杉矶分校）；University of Queensland（昆士兰大学）
The training data in large language models is key to their success, but it also presents privacy and security risks, as it may contain sensitive information. Detecting pre-training data is crucial for mitigating these concerns. Existing methods typically analyze target text in isolation or solely with non-member contexts, overlooking potential insights from simultaneously considering both member and non-member contexts. While previous work suggested that member contexts provide little information due to the minor distributional shift they induce, our analysis reveals that these subtle shifts can be effectively leveraged when contrasted with non-member contexts. In this paper, we propose Con-ReCall, a novel approach that leverages the asymmetric distributional shifts induced by member and non-member contexts through contrastive decoding, amplifying subtle differences to enhance membership inference. Extensive empirical evaluations demonstrate that Con-ReCall achieves state-of-the-art performance on the WikiMIA benchmark and is robust against various text manipulation techniques.
Multi-Level Attention and Contrastive Learning for Enhanced Text Classification with an Optimized Transformer
Stevens Institute of Technology（史蒂文斯理工学院）；San Francisco State University（旧金山州立大学）；University of Minnesota（明尼苏达大学）；The Chinese University of Hong Kong（香港中文大学）；New York University（纽约大学）
Unsupervised Dense Information Retrieval with Contrastive Learning
Meta AI Research（Meta人工智能研究院）；Ecole normale supérieure, PSL University（巴黎文理研究大学高等师范学院）；Inria（法国国家信息与自动化研究所）；Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；University College London（伦敦大学学院）
Unveil Multi-Picture Descriptions for Multilingual Mild Cognitive Impairment Detection via Contrastive Learning
Computer Science, University of Massachusetts, Boston, MA, USA（马萨诸塞大学计算机科学系，波士顿分校）；Computer Science, University of Massachusetts, Lowell, MA, USA（马萨诸塞大学计算机科学系，洛厄尔分校）
