本文指出 LLMs 在解决约束满足与优化问题上的不足，提出基于神经符号方法的 EC AI 平台，结合高性能逻辑推理引擎与 LLMs，在构建、验证和修复解方面显著优于 LLMs。
摘要：
自几年前大型语言模型（LLMs）出现以来，它们常被视为许多 AI 问题的既定解决方案。然而，除了阻碍其在工业界广泛采用的诸多缺陷（如可靠性、成本和速度）外，LLMs 在一类常见的现实世界问题上表现不佳，即约束满足与优化问题。这些问题无处不在，且现有解决方案高度专业化，实施成本高昂。在 Elemental Cognition，我们开发了 EC AI 平台，采用神经符号方法来解决约束满足与优化问题。该平台的核心采用了一个精确且高性能的逻辑推理引擎，并利用 LLMs 进行知识获取和用户交互。该平台支持开发者以自然且简洁的语言指定应用逻辑，同时生成应用用户界面以有效地与用户交互。我们在三个领域对 LLMs 与基于 EC AI 平台构建的系统进行了评估，发现 EC AI 系统在构建有效且最优的解、验证提出的解以及修复无效解方面显著优于 LLMs。
英文摘要：
Since the advent of Large Language Models a few years ago, they have often been considered the de facto solution for many AI problems. However, in addition to the many deficiencies of LLMs that prevent them from broad industry adoption, such as reliability, cost, and speed, there is a whole class of common real world problems that Large Language Models perform poorly on, namely, constraint satisfaction and optimization problems. These problems are ubiquitous and current solutions are highly specialized and expensive to implement. At Elemental Cognition, we developed our EC AI platform which takes a neuro-symbolic approach to solving constraint satisfaction and optimization problems. The platform employs, at its core, a precise and high performance logical reasoning engine, and leverages LLMs for knowledge acquisition and user interaction. This platform supports developers in specifying application logic in natural and concise language while generating application user interfaces to interact with users effectively. We evaluated LLMs against systems built on the EC AI platform in three domains and found the EC AI systems to significantly outperform LLMs on constructing valid and optimal solutions, on validating proposed solutions, and on repairing invalid solutions.
Harbin Institute of Technology（哈尔滨工业大学）；Central South University（中南大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Princeton University（普林斯顿大学）；The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）；ByteDance Seed (China)（字节跳动种子（中国））；Shanghai Jiao Tong University（上海交通大学）
Cross-Lingual Consistency: A Novel Inference Framework for Advancing Reasoning in Large Language Models
Shandong Inspur Artificial Intelligence Research Institute Co., Ltd.（山东浪潮人工智能研究院有限公司）；Shandong Yunhai Guochuang Cloud Computing Equipment Industry Innovation Center Co., Ltd.（山东云海国创云计算装备产业创新中心有限公司）
SAFER: Advancing Safety Alignment via Efficient Ex-Ante Reasoning
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University（浙江大学Hangzhou全球科技创新中心）；Polytechnic Institute, Zhejiang University（浙江大学 polytechnic 院）；School of Software Technology, Zhejiang University（浙江大学软件技术学院）；ZJU-UIUC Institute, Zhejiang University（浙江大学UIUC研究院）
Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback
HCCL, The Chinese University of Hong Kong, Hong Kong, China（香港中文大学人工智能研究中心，香港，中国）；University of Cambridge, Cambridge, United Kingdom（剑桥大学，剑桥，英国）；MMLab, The Chinese University of Hong Kong, Hong Kong, China（香港中文大学人工智能实验室，香港，中国）；Shanghai Artificial Intelligence Laboratory, Shanghai, China（上海人工智能实验室，上海，中国）
Unveiling the Landscape of Clinical Depression Assessment: From Behavioral Signatures to Psychiatric Reasoning
School of Computer Science and Engineering, Central South University（中南大学计算机科学与工程学院）；CoAI Group, DCST, IAI, BNRIST, Tsinghua University（清华大学）；University of International Relations（国际关系大学）；Central China Normal University（华中师范大学）；Lingxin AI（灵心AI）；Yuquan Hospital, Tsinghua University（清华大学友谊医院）
A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined
DRIVE-Health CDT（DRIVE-Health 博士训练中心）；Department of Biostatistics and Health Informatics（生物统计学与健康信息学系）；Institute of Psychiatry, Psychology and Neuroscience（精神病学、心理学与神经科学研究所）；King’s College London（伦敦国王学院）；Cleveland Clinic London（伦敦克利夫兰诊所）
Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learning
Zhejiang University（浙江大学）；Fudan University（复旦大学）；Soochow University（苏州大学）；Shanghai AI Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）
Advancing LLM Reasoning Generalists with Preference Trees
Tsinghua University（清华大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Northeastern University（东北大学）；Renmin University of China（中国人民大学）；BUPT（北京邮电大学）；Tencent（腾讯）
EffiReason-Bench: A Unified Benchmark for Evaluating and Advancing Efficient Reasoning in Large Language Models
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Nanyang Technological University（南洋理工大学）；The Hong Kong University of Science and Technology（香港科技大学）；Singapore Institute of Manufacturing Technology, A*STAR（新加坡制造技术研究所，A*STAR）
ReasVQA: Advancing VideoQA with Imperfect Reasoning Process
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；National Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）
EscapeBench: Towards Advancing Creative Intelligence of Language Model Agents
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Johns Hopkins University（约翰霍普金斯大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Columbia University（哥伦比亚大学）
Understand, Think, and Answer: Advancing Visual Reasoning with Large Multimodal Models
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所基础模型研究中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Peng Cheng Laboratory, Shenzhen, China（鹏城实验室）；Wuhan AI Research, Wuhan, China（武汉人工智能研究所）
Advancing Multi-Step Mathematical Reasoning in Large Language Models through Multi-Layered Self-Reflection with Auto-Prompting
Luiz de Queiroz College of Agriculture University of São Paulo（圣保罗大学农业学院）；School of Engineering and Sciences Tecnologico de Monterrey（蒙特雷技术学院工程与科学学院）；Institute of Mathematics and Computer Sciences University of São Paulo（圣保罗大学数学与计算机科学研究所）
The IOL-AI Challenge: An Open Challenge towards Advancing Linguistic Reasoning
University College London（伦敦大学学院）；Meta；CentraleSupélec（中央高等电力学院）；McGill University（麦吉尔大学）；Cohere Labs（Cohere实验室）；Princeton University（普林斯顿大学）；University of Amsterdam（阿姆斯特丹大学）；Stockholm University（斯德哥尔摩大学）；Faculty of Liberal Arts and Sciences（文理学院）；Universidade Federal de Goiás（戈亚斯联邦大学）
MCERF: Advancing Multimodal LLM Evaluation of Engineering Documentation with Enhanced Retrieval
School of Mechanical, Aerospace, and Manufacturing Engineering, University of Connecticut, Storrs, CT 06269（机械、航空航天与制造工程学院，康涅狄格大学，斯托尔斯，CT 06269）；Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA（机械工程系，麻省理工学院，剑桥，MA 02139，美国）
