Accurately attributing answer text to its source document is crucial for developing a reliable question-answering system. However, attribution for long documents remains largely unexplored. Post-hoc attribution systems are designed to map answer text back to the source document, yet the granularity of this mapping has not been addressed. Furthermore, a critical question arises: What exactly should be attributed? This involves identifying the specific information units within an answer that require grounding. In this paper, we propose and investigate a novel approach to the factual decomposition of generated answers for attribution, employing template-based in-context learning. To accomplish this, we utilize the question and integrate negative sampling during few-shot in-context learning for decomposition. This approach enhances the semantic understanding of both abstractive and extractive answers. We examine the impact of answer decomposition by providing a thorough examination of various attribution approaches, ranging from retrieval-based techniques to LLM-based attributors.
Explanation Bias is a Product: Revealing the Hidden Lexical and Position Preferences in Post-Hoc Feature Attribution
Computational Linguistics and Text Mining Lab, Vrije Universiteit Amsterdam(计算语言学与文本挖掘实验室,荷兰自由大学);TNO–The Netherlands Organization for Applied Scientific Research, The Hague(荷兰应用科学研究院(TNO),海牙);Leiden University Centre for Linguistics (LUCL), Leiden University, Leiden(莱顿大学语言中心(LUCL),莱顿大学,莱顿)
Gender Bias in Explainability: Investigating Performance Disparity in Post-hoc Methods
Technical University of Munich(慕尼黑技术大学);School of Computation, Information and Technology(计算、信息与技术学院);Department of Computer Science(计算机科学系);Technische Universität Berlin(柏林技术大学);BIFOLD – Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究所);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心)
School of Computer and Information Technology, Shanxi University(山西大学计算机与信息学院);Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing Institute of Artificial Intelligence, School of Information Science and Technology, Beijing University of Technology(北京人工智能研究院多媒体与智能软件技术重点实验室,北京理工大学信息科学技术学院);University of Manchester(曼彻斯特大学);Singapore University of Technology and Design(新加坡科技设计大学);ILCC, School of Informatics, University of Edinburgh(爱丁堡大学信息学院ILCC)