Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
Google Research（谷歌研究院）
Bernd Bohnet, Vinh Q. Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Massimiliano Ciaramita, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herz… 展开作者
Bernd Bohnet, Vinh Q. Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Massimiliano Ciaramita, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, Tom Kwiatkowski, Ji Ma, Jianmo Ni, Lierni Sestorain Saralegui, Tal Schuster, William W. Cohen, Michael Collins, Dipanjan Das, Donald Metzler, Slav Petrov, Kellie Webster
Large language models (LLMs) have shown impressive results while requiring little or no direct supervision. Further, there is mounting evidence that LLMs may have potential in information-seeking scenarios. We believe the ability of an LLM to attribute the text that it generates is likely to be crucial in this setting. We formulate and study Attributed QA as a key first step in the development of attributed LLMs. We propose a reproducible evaluation framework for the task and benchmark a broad set of architectures. We take human annotations as a gold standard and show that a correlated automatic metric is suitable for development. Our experimental work gives concrete answers to two key questions (How to measure attribution?, and How well do current state-of-the-art methods perform on attribution?), and give some hints as to how to address a third (How to build LLMs with attribution?).
Can LLMs Evaluate Complex Attribution in QA? Automatic Benchmarking using Knowledge Graphs
School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Key Laboratory of New Generation Artificial Intelligence Technology and its Interdisciplinary Applications (Southeast University), Ministry of Education（教育部新一代人工智能技术及其交叉应用关键实验室（东南大学））；The University of Manchester（曼彻斯特大学）；The University of Edinburgh（爱丁堡大学）；Law and Innovation Lab, School of Law, Southeast University（东南大学法学院创新与法律实验室）；State Key Lab. for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）；The Chinese University of Hong Kong（香港中文大学）；Monash University（莫纳什大学）
Unifying Text Semantics and Graph Structures for Temporal Text-attributed Graphs with Large Language Models
Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University（上海数据科学 key laboratory，计算机科学学院，复旦大学）；School of Management, Fudan University（管理学院，复旦大学）；Tencent Weixin Group（腾讯微信集团）；IFM Lab, University of California, Davis（加州大学戴维斯分校IFM实验室）
Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications
The University of New South Wales（新南威尔士大学）；The University of Technology Sydney（技术大学悉尼分校）；University of Technology Sydney（技术大学悉尼分校）；Duke University（杜克大学）
School of Computer and Information Technology, Shanxi University（山西大学计算机与信息学院）；Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing Institute of Artificial Intelligence, School of Information Science and Technology, Beijing University of Technology（北京人工智能研究院多媒体与智能软件技术重点实验室，北京理工大学信息科学技术学院）；University of Manchester（曼彻斯特大学）；Singapore University of Technology and Design（新加坡科技设计大学）；ILCC, School of Informatics, University of Edinburgh（爱丁堡大学信息学院ILCC）
Context Modeling with Evidence Filter for Multiple Choice Question Answering
Singapore Management University（新加坡管理大学）；Nanyang Technological University（南洋理工大学）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）；Institute of High Performance Computing, A*STAR（新加坡科技研究局高性能计算研究院）
Exploring Generative Process Reward Modeling for Semi-Structured Data: A Case Study of Table Question Answering
University of Augsburg（奥格斯堡大学）；Institut für Maschinelle Sprachverarbeitung, University of Stuttgart（斯图加特大学机械语言处理研究所）；Bosch Center for Artificial Intelligence（博世人工智能中心）
PathReasoner: Modeling Reasoning Path with Equivalent Extension for Logical Question Answering
School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；Ministry of Education Key Laboratory of Intelligent Networks and Network Security（教育部智能网络与网络安全重点实验室）；Shaanxi Province Key Laboratory of Big Data Knowledge Engineering（陕西省大数据知识工程重点实验室）
BELIEF: Structured Evidence Modeling and Uncertainty-Aware Fusion for Biomedical Question Answering
School of Computer Science and Technology, Zhejiang University of Science and Technology（浙江理工大学计算机科学与技术学院）；College of Artificial Intelligence, Zhejiang University（浙江大学人工智能学院）；Zhejiang Key Laboratory of Biomedical Intelligent Computing Technology, Zhejiang University of Science and Technology（浙江理工大学生物医学智能计算技术重点实验室）
HeteroQA: Learning towards Question-and-Answering through Multiple Information Sources via Heterogeneous Graph Modeling
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；Ant Group（蚂蚁集团）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
Probing Emergent Semantics in Predictive Agents via Question Answering
Georgia Institute of Technology（佐治亚理工学院）；DeepMind（深度思维）
最近的研究表明，预测建模可以赋予智能体丰富的环境知识，提高他们在复杂环境中的行动能力。我们提出问答作为一种通用范式来解码和理解这种代理开发的表示，将我们的方法应用于最近的两种预测建模方法-动作条件CPC（Guo et al.，2018）和SimCore（Gregor等人，2019年）。在一个视觉丰富的3D环境中训练具有这些预测目标的智能体，其中包含各种各样的对象，颜色，形状和空间配置，我们用合成（英语）问题探测它们的内部状态表示，而不需要将梯度从问答解码器反向传播到智能体中。当以这种方式探索时，不同代理的表现表明，它们学会了从物理环境中编码关于物体，属性和空间关系的事实和看似组成的信息。我们的方法是直观的，即人类可以很容易地解释模型的响应，而不是检查连续向量，并且模型不可知，即适用于任何建模方法。通过揭示主体在学习过程中获得的对象、数量、属性和关系的隐含知识，问题条件主体探测可以刺激更强预测性学习目标的设计和发展。
