How Do LLMs Encode Scientific Quality? An Empirical Study Using Monosemantic Features from Sparse Autoencoders
Knowledge Media Institute, The Open University, Walton Hall, Milton Keynes, MK7 6AA, United Kingdom(开放大学知识媒体学院);Department of Business and Law, University of Milano Bicocca, Milan, IT(米兰比可卡大学商学院)
Michael McCoubrey, Angelo Salatino, Francesco Osborne, Enrico Motta
备注:
Presented at SESAME 2025: Smarter Extraction of ScholArly MEtadata using Knowledge Graphs and Language Models, @ JCDL 2025
In recent years, there has been a growing use of generative AI, and large language models (LLMs) in particular, to support both the assessment and generation of scientific work. Although some studies have shown that LLMs can, to a certain extent, evaluate research according to perceived quality, our understanding of the internal mechanisms that enable this capability remains limited. This paper presents the first study that investigates how LLMs encode the concept of scientific quality through relevant monosemantic features extracted using sparse autoencoders. We derive such features under different experimental settings and assess their ability to serve as predictors across three tasks related to research quality: predicting citation count, journal SJR, and journal h-index. The results indicate that LLMs encode features associated with multiple dimensions of scientific quality. In particular, we identify four recurring types of features that capture key aspects of how research quality is represented: 1) features reflecting research methodologies; 2) features related to publication type, with literature reviews typically exhibiting higher impact; 3) features associated with high-impact research fields and technologies; and 4) features corresponding to specific scientific jargons. These findings represent an important step toward understanding how LLMs encapsulate concepts related to research quality.
Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models
Technical University of Munich(慕尼黑技术大学);Helmholtz Munich(亥姆霍兹慕尼黑);Munich Center for Machine Learning(慕尼黑机器学习中心);Munich Data Science Institute(慕尼黑数据科学研究所);University of Tübingen(图宾根大学);University of Copenhagen(哥本哈根大学)
Computer Science Department, TU Darmstadt(图宾根大学计算机科学系);Lab1141;Aleph Alpha Research(Aleph Alpha研究);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);CERTAIN;Centre of Cognitive Science, TU Darmstadt(图宾根大学认知科学中心)
CRISP: Persistent Concept Unlearning via Sparse Autoencoders
Technion – Israel Institute of Technology(技术ion-以色列理工学院);Boston University(波士顿大学);University of Zagreb, FER(Zagreb大学,FER);Kempner Institute, Harvard University(哈佛大学 Kempner研究所)
Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework
Technische Universität Berlin(柏林技术大学);BIFOLD;UMI Lab(UMI实验室);Fraunhofer Heinrich-Hertz-Institute(弗劳恩霍夫海因里希-赫兹研究所);ETH AI Center(苏黎世联邦理工学院人工智能中心);Universität Potsdam(波茨坦大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心)
Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework
Alibaba Group(阿里巴巴集团);The State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学区块链与数据安全国家重点实验室);Language Technology Lab, University of Cambridge(剑桥大学语言技术实验室);Renmin University of China(中国人民大学)
From Atoms to Trees: Building a Structured Feature Forest with Hierarchical Sparse Autoencoders
School of Mathematical Science, Peking University, Beijing, China(北京大学数学科学学院);Beijing International Center for Mathematical Research(北京国际数学研究中心);the New Cornerstone Science Laboratory, Peking University, Beijing, China(北京大学新基石科学实验室);Center for Machine Learning Research, Peking University, Beijing, China(北京大学机器学习研究中心);School of Information and Electronics, Beijing Institute of Technology, Beijing, China(北京理工大学信息与电子学院);Institute for Theoretical Sciences, Westlake University, Hangzhou, China(西湖大学理论科学研究院);School of Science, Westlake University, Hangzhou, China(西湖大学理学院);School of Informatics, University of Edinburgh, Edinburgh, UK(爱丁堡大学信息学院)
Exploring Task Performance with Interpretable Models via Sparse Auto-Encoders
The University of Sheffield(谢菲尔德大学);Tianjin University(天津大学);University of Manchester(曼彻斯特大学);Kyushu University(九州大学);City University of Hong Kong(香港城市大学)
Filter-based Discriminative Autoencoders for Children Speech Recognition
Institute of Information Science, Academia Sinica(中央研究院信息科学研究所);Research Center for Information Technology Innovation, Academia Sinica(中央研究院信息科技创新研究中心)