Contextual Attention Modulation: Towards Efficient Multi-Task Adaptation in Large Language Models
SCSE, Beihang University(北京航空航天大学系统工程学院);ERC of ACAT, MOE(教育部人工智能教育联合体);City University of Hong Kong(香港城市大学);Huawei Technologies Ltd.(华为技术有限公司);Key Lab of DIM, MIIT(信息通信技术重点实验室);SEM, Beihang University(北京航空航天大学电子机械学院);Zhejiang University of Technology(浙江工业大学)
Large Language Models (LLMs) possess remarkable generalization capabilities but struggle with multi-task adaptation, particularly in balancing knowledge retention with task-specific specialization. Conventional fine-tuning methods suffer from catastrophic forgetting and substantial resource consumption, while existing parameter-efficient methods perform suboptimally in complex multi-task scenarios. To address this, we propose Contextual Attention Modulation (CAM), a novel mechanism that dynamically modulates the representations of self-attention modules in LLMs. CAM enhances task-specific features while preserving general knowledge, thereby facilitating more effective and efficient adaptation. For effective multi-task adaptation, CAM is integrated into our Hybrid Contextual Attention Modulation (HyCAM) framework, which combines a shared, full-parameter CAM module with multiple specialized, lightweight CAM modules, enhanced by a dynamic routing strategy for adaptive knowledge fusion. Extensive experiments on heterogeneous tasks, including question answering, code generation, and logical reasoning, demonstrate that our approach significantly outperforms existing approaches, achieving an average performance improvement of 3.65%. The implemented code and data are available to ease reproducibility at https://github.com/Applied-Machine-Learning-Lab/HyCAM.
AI Knows When It's Being Watched: Functional Strategic Action and Contextual Register Modulation in Large Language Models
Center for Applied Communication Research (CICA)(应用沟通研究中心);Human & NonHuman Communication Laboratory(人类与非人类沟通实验室);Faculty of Communication(传播学院);Universidad Anáhuac México(墨西哥安纳胡阿克大学)
Culinary Crossroads: A RAG Framework for Enhancing Diversity in Cross-Cultural Recipe Adaptation
Aarhus University(奥胡斯大学);University of Copenhagen(哥本哈根大学);Dept. of Computer Science and Artificial Intelligence, University of Granada(格拉纳达大学计算机科学与人工智能系);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
EventOD: Event-Aware OD Flow Generation via LLM-Guided Semantic Modulation
Department of Electronic Engineering, Tsinghua University(清华大学电子工程系);Zhongguancun Academy(中关村科学城研究院);Singapore-MIT Alliance for Research and Technology(新加坡-麻省理工学院科研与技术联盟);Xiuzhong College, Tsinghua University(清华大学致理书院);Department of Automation, Central South University(中南大学自动化学院)
Threshold Modulation for Online Test-Time Adaptation of Spiking Neural Networks
Department of CSE, Southern University of Science and Technology(南方科技大学计算机科学与工程系);College of Mathematics and Statistics, Chongqing University(重庆大学数学与统计学院);ACS Laboratory, Huawei Technologies Co., Ltd.(华为技术有限公司ACS实验室)
TAVAE: A VAE with Adaptable Priors Explains Contextual Modulation in the Visual Cortex
Department of Computational Sciences(计算科学系);HUN-REN Wigner Research Centre for Physics(霍恩-伦维格物理研究中心);Princeton Neuroscience Institute(普林斯顿神经科学研究所);Princeton University(普林斯顿大学);Center for Molecular and Behavioral Neuroscience(分子与行为神经科学中心);Rutgers University(罗格斯大学)