DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learning
National University of Singapore(新加坡国立大学);Singapore-MIT Alliance for Research and Technology(新加坡-麻省理工学院研究与技术联盟);Institute for Infocomm Research, A*STAR(新加坡科技研究局信息通信研究所);CSAIL, MIT(麻省理工学院计算机科学与人工智能实验室)
In-context learning (ICL) allows transformer-based language models that are pre-trained on general text to quickly learn a specific task with a few "task demonstrations" without updating their parameters, significantly boosting their flexibility and generality. ICL possesses many distinct characteristics from conventional machine learning, thereby requiring new approaches to interpret this learning paradigm. Taking the viewpoint of recent works showing that transformers learn in context by formulating an internal optimizer, we propose an influence function-based attribution technique, DETAIL, that addresses the specific characteristics of ICL. We empirically verify the effectiveness of our approach for demonstration attribution while being computationally efficient. Leveraging the results, we then show how DETAIL can help improve model performance in real-world scenarios through demonstration reordering and curation. Finally, we experimentally prove the wide applicability of DETAIL by showing our attribution scores obtained on white-box models are transferable to black-box models in improving model performance.
CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Jinesis AI Lab(Jinesis人工智能实验室);University of Toronto(多伦多大学);Vector Institute(向量研究所);CMU(卡内基梅隆大学);EuroSafeAI;ELLIS Institute Tübingen(图宾根ELLIS研究所)
FitCF: A Framework for Automatic Feature Importance-guided Counterfactual Example Generation
Quality and Usability Lab, Technische Universität Berlin(柏林技术大学质量与可用性实验室);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Deutsche Telekom(德国电信);Saarland Informatics Campus(萨尔州信息学校区);Centre for European Research in Trusted AI (CERTAIN)(可信人工智能欧洲研究中心);BIFOLD – Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究院)