Understanding transformer-based language models is becoming increasingly crucial, particularly as they play pivotal roles in advancing towards artificial general intelligence. However, language model research faces significant challenges, especially for academic research groups with constrained resources. These challenges include complex data structures, unknown target functions, high computational costs and memory requirements, and a lack of interpretability in the inference process, etc. Drawing a parallel to the use of simple models in scientific research, we propose the concept of an anchor function. This is a type of benchmark function designed for studying language models in learning tasks that follow an "anchor-key" pattern. By utilizing the concept of an anchor function, we can construct a series of functions to simulate various language tasks. The anchor function plays a role analogous to that of mice in diabetes research, particularly suitable for academic research. We demonstrate the utility of the anchor function with an example, revealing two basic operations by attention structures in language models: shifting tokens and broadcasting one token from one position to many positions. These operations are also commonly observed in large language models. The anchor function framework, therefore, opens up a series of valuable and accessible research questions for further exploration, especially for theoretical study.
AnchorBench: A Multi-Pathway Benchmark for the Anchoring Effect in LLMs
Saarland University（萨尔大学）；Hamburg University of Technology（汉堡工业大学）；Helmholtz-Zentrum Hereon（亥姆霍兹中心赫伦）；German Research Centre for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））
SparseEval: Efficient Evaluation of Large Language Models by Sparse Optimization
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Microsoft Research Asia（微软亚洲研究院）
Provably End-to-end Label-Noise Learning without Anchor Points
University of New South Wales（新南威尔士大学）；University of Sydney（悉尼大学）；Hong Kong Baptist University（香港浸会大学）；RIKEN AIP（日本理化学研究所创新智能综合研究中心）；University of Tokyo（东京大学）
SinoGlyphBench: A Diagnostic Benchmark for Chinese Glyph-Level Obfuscation in Language-Model Moderation
East China Normal University（华东师范大学）；New York University Shanghai（上海纽约大学）；University of Liverpool（利物浦大学）；Singapore University of Technology and Design（新加坡科技设计大学）；Eastern Institute of Technology (Ningbo)（宁波东方理工大学）
When Confidence Misleads: Suffix Anchoring and Anchor-Proximity Confidence Modulation for Diffusion Language Models
RICS（智能研究学院）；AIIS（人工智能研究所）；IPAI（人工智能研究所）；Department of Intelligence and Information（智能与信息系）；Daegu Gyeongbuk Institute of Science and Technology（大邱庆州市科学技术院）
