认知系统通常需要人类将问题定义转化为某种规范,以便认知系统利用该规范尝试解决问题或执行任务。在本文中,我们说明大型语言模型(LLM)可用于将以自然语言定义的问题类别映射为半形式化规范,随后现有推理与学习系统可利用该规范解决该问题类别的实例。我们提出了支持 LLM 的认知任务分析智能体的设计。该系统通过 LLM 智能体实现,为以自然语言指定的任务生成问题空间定义。LLM 提示源自 AI 文献中对问题空间的定义以及通用问题解决策略(Polya 的《How to Solve It》)。随后,认知系统可以使用该问题空间规范,应用领域通用的问题解决策略(诸如搜索之类的“弱方法”),来解决该问题类别中的多个问题实例。这一结果虽然尚属初步,但表明通过去除问题表述中的人工中介环节,同时保留认知系统的核心能力(如稳健推理和在线学习),有望加快认知系统研究。
英文摘要:
Cognitive systems generally require a human to translate a problem definition into some specification that the cognitive system can use to attempt to solve the problem or perform the task. In this paper, we illustrate that large language models (LLMs) can be utilized to map a problem class, defined in natural language, into a semi-formal specification that can then be utilized by an existing reasoning and learning system to solve instances from the problem class. We present the design of LLM-enabled cognitive task analyst agent(s). Implemented with LLM agents, this system produces a definition of problem spaces for tasks specified in natural language. LLM prompts are derived from the definition of problem spaces in the AI literature and general problem-solving strategies (Polya's How to Solve It). A cognitive system can then use the problem-space specification, applying domain-general problem solving strategies ("weak methods" such as search), to solve multiple instances of problems from the problem class. This result, while preliminary, suggests the potential for speeding cognitive systems research via disintermediation of problem formulation while also retaining core capabilities of cognitive systems, such as robust inference and online learning.
LINKED: Eliciting, Filtering and Integrating Knowledge in Large Language Model for Commonsense Reasoning
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Eliciting Causal Abilities in Large Language Models for Reasoning Tasks
BNU-HKBU United International College(北师港浸大);Hong Kong Baptist University(香港浸会大学);Beijing Normal University at Zhuhai(北京师范大学珠海校区);The Chinese University of Hong Kong(香港中文大学)
Eliciting Critical Reasoning in Retrieval-Augmented Language Models via Contrastive Explanations
Idiap Research Institute(Idiap研究机构);Department of Computer Science, University of Manchester(曼彻斯特大学计算机科学系);National Biomarker Centre (NBC), CRUK Manchester Institute(国家生物标志物中心(NBC),CRUK曼彻斯特研究所)
Eliciting Trustworthiness Priors of Large Language Models via Economic Games
University of Hong Kong(香港大学);The University of Hong Kong(香港大学);Peking University(北京大学);School of Psychological and Cognitive Sciences, Peking University(北京大学心理与认知科学学院);Beijing Key Laboratory of Behavior and Mental Health, Peking University(北京大学行为与心理健康重点实验室);IDG/McGovern Institute for Brain Research, Peking University(北京大学脑科学研究院);Peking-Tsinghua Center for Life Sciences, Peking University(北京大学-清华大学生命科学中心);Key Laboratory of Machine Perception, Ministry of Education, China(教育部机器感知重点实验室)
RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models
Beijing University of Posts and Telecommunications(北京邮电大学);The Hong Kong University of Science and Technology(香港科技大学);Beihang University(北京航空航天大学);University of the Chinese Academy of Sciences(中国科学院大学);ETH Zürich(苏黎世联邦理工学院);The Hong Kong Polytechnic University(香港理工大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Shanghai AI Lab(上海人工智能实验室)
AgentGroupChat: An Interactive Group Chat Simulacra For Better Eliciting Emergent Behavior
Fudan University(复旦大学);Rhine AI(莱茵人工智能公司);Shanghai University of Engineering and Science(上海工程技术大学);Xiaohongshu Inc.(小红书公司);Fudan-Aishu Cognitive Intelligence Joint Research Center(复旦-爱数认知智能联合研究中心)
SimLens for Early Exit in Large Language Models: Eliciting Accurate Latent Predictions with One More Token
Institute of Neuroscience, State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences(中国科学院脑科学与智能技术卓越创新中心神经科学研究所,脑认知与脑启发智能技术国家重点实验室,脑科学与智能技术卓越创新中心);School of Future Technology, University of Chinese Academy of Sciences(中国科学院大学未来技术学院)
TO-GATE: Clarifying Questions and Summarizing Responses with Trajectory Optimization for Eliciting Human Preference
School of Information Science and Engineering, Yunnan University, China(云南大学信息科学与工程学院);Yunnan Key Laboratory of Intelligent Systems and Computing, China(云南省智能系统与计算重点实验室)
Re-Reading Improves Reasoning in Large Language Models
Institute of Information Engineering, CAS(中国科学院信息工程研究所);SKLSDE Lab, Beihang University(北京航空航天大学);University of Technology Sydney(悉尼科技大学);Microsoft Corporation(微软公司)