出乎意料:相似度得分的对比效应
Taken by Surprise: Contrast effect for Similarity Scores
AI总结:
针对现有相似度得分未考虑对象集合分布的问题,提出体现人类感知对比效应的surprise score指标,在零/少样本分类等任务上性能较余弦相似度提升10-15%。
摘要:
准确评估对象向量嵌入的相似度对自然语言处理、信息检索和分类任务至关重要。常用的相似度得分(如余弦相似度)基于嵌入向量对计算,未考虑对象所在整体集合的分布情况。人类对对象相似度的感知很大程度上取决于对象出现的上下文。本研究提出了$\textit{surprise score}$(惊喜得分),这是一种经整体集合归一化的相似度指标,体现了人类感知的对比效应,能显著提升零样本和少样本文档分类任务的分类性能。该得分量化了在两个元素间观测到特定相似度时,相对于整体集合内两两相似度的意外程度。我们在零样本/少样本分类和聚类任务上评估了该指标,发现其性能通常比原始余弦相似度高10-15%。我们的代码可在https://github.com/MeetElise/surprise-similarity获取。
英文摘要:
Accurately evaluating the similarity of object vector embeddings is of critical importance for natural language processing, information retrieval and classification tasks. Popular similarity scores (e.g cosine similarity) are based on pairs of embedding vectors and disregard the distribution of the ensemble from which objects are drawn. Human perception of object similarity significantly depends on the context in which the objects appear. In this work we propose the $\textit{surprise score}$, an ensemble-normalized similarity metric that encapsulates the contrast effect of human perception and significantly improves the classification performance on zero- and few-shot document classification tasks. This score quantifies the surprise to find a given similarity between two elements relative to the pairwise ensemble similarities. We evaluate this metric on zero/few shot classification and clustering tasks and typically find 10-15 % better performance compared to raw cosine similarity. Our code is available at https://github.com/MeetElise/surprise-similarity.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。