Active Few-Shot Learning (AFSL) adapts LLMs to specialized domains by identifying the most valuable unlabeled samples for annotation and use as few-shot demonstrations, effectively reducing human annotation costs while promoting high performance. However, existing methods typically rely on output-level signals for sample identification, such as predictive entropy or semantic similarities with test-time data based on external embeddings, which often overlook models' internal dynamics, which could pinpoint specific knowledge gaps. To bridge this gap, we propose NeuFS, a Neuron-Aware Active Few-Shot Learning framework that shifts the selection paradigm from output-level proxies to models' internal dynamics. NeuFS utilizes neuron activation patterns to represent sample directly, and includes a dual-criteria selection strategy that: (1) ensures few-shot sample diversity with neuron patterns for broader example coverage, while (2) prioritizing on identifying informative and challenging few-shot samples LLMs tend to hallucinate by quantifying neuron consensus. Experiments on three datasets demonstrate that NeuFS excels in both reasoning and text classification tasks, outperforming existing AFSL baselines. Ablation studies further highlight that internal neuron activations provide a more principled and effective selection signal than external embeddings, validating the superiority of the proposed NeuFS.
The Achilles' Heel of LLMs: How Altering a Handful of Neurons Can Cripple Language Abilities
Center for Applied Statistics, School of Statistics, Renmin University of China（中国人民大学统计学院应用统计中心）；Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing（未来区块链与隐私计算北京先进创新中心）；School of Artificial Intelligence, Beihang University（北航人工智能学院）
Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons
Department of Computer Science and Technology, BNRist（计算机科学与技术系，BNRist）；Shenzhen International Graduate School（深圳国际研究生院）；KIRC, Institute for Artificial Intelligence, Tsinghua University（人工智能研究院，清华大学）
ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs
Tsinghua University（清华大学）；Shanghai Jiao Tong University（上海交通大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Renmin University of China（中国人民大学）
Linguistic Neuron Overlap Patterns to Facilitate Cross-lingual Transfer on Low-resource Languages
School of Information Science and Technology, Beijing Foreign Studies University（信息科学与技术学院，北京外国语大学）；Department of Informatics, King’s College London（信息学院，伦敦国王学院）
Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing Wenge Technology Co., Ltd（北京文格科技有限公司）
The University of Tokyo（东京大学）；Google Research（谷歌研究院）
Pretrained large language models (LLMs) are widely used in many sub-fields of natural language processing (NLP) and generally known as excellent few-shot learners with task-specific exemplars. Notably, chain of thought (CoT) prompting, a recent technique for eliciting complex multi-step reasoning through step-by-step answer examples, achieved the state-of-the-art performances in arithmetics and symbolic reasoning, difficult system-2 tasks that do not follow the standard scaling laws for LLMs. While these successes are often attributed to LLMs' ability for few-shot learning, we show that LLMs a…
Very large language models (LLMs), such as GPT-3 and Codex have achieved state-of-the-art performance on several natural-language tasks, and show great promise also for code. A particularly exciting aspect of LLMs is their knack for few-shot and zero-shot learning: they can learn to perform a task with very few examples. Few-shotting has particular synergies in software engineering, where there are a lot of phenomena (identifier names, APIs, terminology, coding patterns) that are known to be highly project-specific. However, project-specific data can be quite limited, especially early in the h…
Dissecting Role Cognition in Medical LLMs via Neuronal Ablation
School of Information, Renmin University of China（中国人民大学信息学院）；Center of Machine Learning Research, Peking University（北京大学机器学习研究中心）；School of Artificial Intelligence, Shanghai Jiaotong University（上海交通大学人工智能学院）；Health Informatics Committee of the China Society for Scientific and Technical Information, Renmin University of China（中国科学技术信息学会健康信息技术委员会，中国人民大学）；Institute for Advanced Algorithms Research (IAAR), Shanghai, China（先进算法研究所（IAAR），上海，中国）；Memtensor Research Center, Shanghai, China（Memtensor研究中心，上海，中国）
Meta-Prompting for Automating Zero-shot Visual Recognition with LLMs
ICG, TU Graz（格拉茨工业大学ICG研究所）；CDL-EML（CDL-EML中心）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；JKU（约翰内斯开普勒大学）；IBM Research（IBM研究院）；Weizmann Institute of Science（魏茨曼科学研究所）；University of Bonn（波恩大学）
Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Models
School of Automation, Northwestern Polytechnical University, Xi’an, China（自动化学院，西北工业大学，西安，中国）；School of Computing, University of Georgia, Athens, USA（计算机学院，佐治亚大学，亚特兰大，美国）；School of Physics and Information Technology, Shaanxi Normal University, Xi’an, China（物理与信息技术学院，陕西师范大学，西安，中国）
NeuPAT: Neuron-aware Plasticity Allocation Tuning for Language-Preserving MLLMs
College of Intelligent Robotics and Advanced Manufacturing, Fudan University（复旦大学智能机器人与先进制造学院）；Zhongguancun Academy（中关村学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Tianjin University（天津大学）；Nankai University（南开大学）
LLMs-based Few-Shot Disease Predictions using EHR: A Novel Approach Combining Predictive Agent Reasoning and Critical Agent Instruction
Emory University（埃默里大学）；University of Washington（华盛顿大学）；UCSD（加利福尼亚大学圣迭戈分校）；Rollins School of Public Health, Emory University（埃默里大学罗林斯公共卫生学院）；School of Medicine, Emory University（埃默里大学医学院）
The Emergence of Abstract Thought in Large Language Models Beyond Any Language
National University of Singapore（新加坡国立大学）；Salesforce AI Research（Salesforce AI研究院）；Peking University（北京大学）；Singapore University of Technology and Design（新加坡科技设计大学）
Your AI, On a Dial: Controlling Investment Bias in LLMs with a Single Neuron
Hankuk University of Foreign Studies（韩国外国语大学）；UNIST（蔚山科学技术院）；LG AI Research（LG AI研究院）；Hanwha Life（韩华生命保险）；University of Florida（佛罗里达大学）；Massachusetts Institute of Technology（麻省理工学院）；LinqAlpha
Cracking Factual Knowledge: A Comprehensive Analysis of Degenerate Knowledge Neurons in Large Language Models
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Unisound（云知声）；The Laboratory of Cognition and Decision Intelligence for Complex Systems（复杂系统认知与决策智能实验室）
