With the rise of Large Language Models (LLMs) in recent years, abundant new opportunities are emerging, but also new challenges, among which contamination is quickly becoming critical. Business applications and fundraising in Artificial Intelligence (AI) have reached a scale at which a few percentage points gained on popular question-answering benchmarks could translate into dozens of millions of dollars, placing high pressure on model integrity. At the same time, it is becoming harder and harder to keep track of the data that LLMs have seen; if not impossible with closed-source models like GPT-4 and Claude-3 not divulging any information on the training set. As a result, contamination becomes a major issue: LLMs' performance may not be reliable anymore, as the high performance may be at least partly due to their previous exposure to the data. This limitation jeopardizes real capability improvement in the field of NLP, yet, there remains a lack of methods on how to efficiently detect contamination. In this paper, we survey all recent work on contamination detection with LLMs, analyzing their methodologies and use cases to shed light on the appropriate usage of contamination detection methods. Our work calls the NLP research community's attention into systematically taking into account contamination bias in LLM evaluation.
A Survey on Data Contamination for Large Language Models
College of Software, Jilin University(吉林大学软件学院);School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China(知识驱动人机智能工程研究中心,教育部,中国);International Center of Future Science, Jilin University(未来科学国际中心,吉林大学)
Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluation
Columbia University(哥伦比亚大学);National University of Singapore(国立新加坡大学);University of California, Riverside(加州大学河滨分校);University of Southern California(南加州大学);The Ohio State University(俄亥俄州立大学);Peking University(北京大学);Tsinghua University(清华大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
A Comprehensive Survey on Aspect Based Sentiment Analysis
Vellore Institute Of Technology(韦洛尔理工大学);School of Computer Science and Engineering- SCOPE(计算机科学与工程学院-SCOPE)
基于方面的情感分析(Aspect Based Sentiment Analysis,ABSA)是自然语言处理的一个分支,它主要是将数据分成方面,并最终提取情感信息。已知ABSA比一般的情感分析提供更多关于上下文的信息。在这项研究中,我们的目的是探索各种方法,同时执行ABSA,并提供一个比较研究。这份调查报告深入讨论了各种解决方案,并对它们进行了比较。并方便地分为几个部分,以获得对该过程的整体看法。