提出 L3GO，一种基于三维思维链的语言智能体，通过大语言模型在 3D 仿真环境中试错组合来生成非常规物体，并在新基准 UFO 及 ShapeNet 上超越现有模型。
摘要：
诸如 DALL-E 3 和 Stable Diffusion-XL 等基于扩散的图像生成模型，在生成具有逼真且独特构图的图像方面展现出卓越能力。然而，这些模型在精确推理物体的物理和空间配置时并不稳健，尤其是当被指示使用非常规（从而属于分布外）的描述时，例如“一把有五条腿的椅子”。在本文中，我们提出了一种具有三维思维链的语言智能体（L3GO），这是一种推理时方法，能够对当前数据驱动的扩散模型难以处理的非常规物体进行基于部件的 3D 网格生成推理。更具体地说，我们使用大语言模型作为智能体，在 3D 仿真环境中通过试错来组合所需的物体。为了促进我们的研究，我们开发了一个新基准 Unconventionally Feasible Objects (UFO)，以及 SimpleBlenv，这是一个构建在 Blender 之上的包装环境，语言智能体可以通过 API 调用在其中构建和组合原子构建块。人类和自动 GPT-4V 评估表明，我们的方法在 ShapeNet 上的 3D 网格生成方面超越了标准 GPT-4 和其他语言智能体（例如 ReAct 和 Reflexion）。此外，当在我们的 UFO 基准上进行测试时，基于人类评估，我们的方法优于其他最先进的文本到 2D 图像和文本到 3D 模型。
英文摘要：
Diffusion-based image generation models such as DALL-E 3 and Stable Diffusion-XL demonstrate remarkable capabilities in generating images with realistic and unique compositions. Yet, these models are not robust in precisely reasoning about physical and spatial configurations of objects, especially when instructed with unconventional, thereby out-of-distribution descriptions, such as "a chair with five legs". In this paper, we propose a language agent with chain-of-3D-thoughts (L3GO), an inference-time approach that can reason about part-based 3D mesh generation of unconventional objects that current data-driven diffusion models struggle with. More concretely, we use large language models as agents to compose a desired object via trial-and-error within the 3D simulation environment. To facilitate our investigation, we develop a new benchmark, Unconventionally Feasible Objects (UFO), as well as SimpleBlenv, a wrapper environment built on top of Blender where language agents can build and compose atomic building blocks via API calls. Human and automatic GPT-4V evaluations show that our approach surpasses the standard GPT-4 and other language agents (e.g., ReAct and Reflexion) for 3D mesh generation on ShapeNet. Moreover, when tested on our UFO benchmark, our approach outperforms other state-of-the-art text-to-2D image and text-to-3D models based on human evaluation.
Generating Chain-of-Thoughts with a Pairwise-Comparison Approach to Searching for the Most Promising Intermediate Thought
Center for Advanced Intelligence Project, RIKEN（理化学研究所先进智能项目中心）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Graduate School of Frontier Sciences, The University of Tokyo（东京大学前沿科学研究生院）
OptiTree: Hierarchical Thoughts Generation with Tree Search for LLM Optimization Modeling
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China（脑启发式感知与认知联合实验室，中国科学技术大学）；Noah’s Ark Lab, Huawei Technologies（华为诺亚实验室）；Tianjin University（天津大学）
keqing: knowledge-based question answering is a nature chain-of-thought mentor of LLM
National Key Laboratory of Radar Signal Processing（雷达信号处理国家重点实验室）；Xidian University（西安电子科技大学）；School of Computer Science and Engineering（计算机科学与工程学院）；Nanyang Technological University（南洋理工大学）
Reason Like a Radiologist: Chain-of-Thought and Reinforcement Learning for Verifiable Report Generation
Imperial College London（帝国理工学院）；Sun Yat-sen University（中山大学）；HKUST (GZ)（香港科技大学（广州））；Academy of Athens（雅典科学院）；Athena Research Centre（雅典娜研究中心）；Royal Brompton Hospital（皇家布朗普顿医院）
A Framework for Collaborating a Large Language Model Tool in Brainstorming for Triggering Creative Thoughts
R.B. Annis School of Engineering（R.B.安尼斯工程学院）；University of Indianapolis（印第安纳波利斯大学）；School of Journalism and Strategic Communication（新闻与战略传播学院）；Ball State University（鲍尔州立大学）
Focused Chain-of-Thought: Efficient LLM Reasoning via Structured Input Information
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Technical University of Darmstadt（德累斯顿技术大学）；University of Kassel（卡塞尔大学）；TU Dortmund University（Dortmund技术大学）；TU Center for Trustworthy Data Science and Security, University Alliance Ruhr（TU可信数据科学与安全中心，鲁尔大学联盟）；Hessian Center for AI (Hessian.AI)（黑森人工智能中心（黑森.AI））；Centre for Cognitive Science, Technical University of Darmstadt（认知科学中心，德累斯顿技术大学）
Think Consistently, Reason Efficiently: Energy-Based Calibration for Implicit Chain-of-Thought
University of Oxford（牛津大学）；Tsinghua University（清华大学）；Tencent（腾讯）；Southern University of Science and Technology（南方科技大学）；National University of Singapore（新加坡国立大学）
Controllable Navigation Instruction Generation with Chain of Thought Prompting
School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual intervention
Hong Kong University of Science and Technology（香港科技大学）；Peking University（北京大学）；University of Hong Kong（香港大学）；NUS（南洋理工大学）；University of Central Florida（佛罗里达大学）；Everlyn AI
Training Multimodal Large Reasoning Models Needs Better Thoughts: A Three-Stage Framework for Long Chain-of-Thought Synthesis and Selection
School of Computer Science and Engineering（计算机科学与工程学院）；Key Laboratory of Computer Network and Information Integration (SEU), Ministry of Education（计算机网络与信息集成重点实验室（SEU））
A Theory of Learning with Autoregressive Chain of Thought
Toyota Technological Institute at Chicago（丰田技术研究所（芝加哥））；Weizmann Institute of Science（魏兹曼科学研究院）；Microsoft Research, NYC（微软研究院（纽约））；University of Pennsylvania（宾夕法尼亚大学）；Yale University（耶鲁大学）
Evolving in Thought Space: Training a Small Model at Test Time Unlocks Better Discoveries
Massachusetts Institute of Technology（麻省理工学院）；Hong Kong Polytechnic University（香港理工大学）；ByteDance Inc.（字节跳动公司）；National University of Singapore（新加坡国立大学）；Stanford University（斯坦福大学）；University of Washington（华盛顿大学）；University of California, Berkeley（加州大学伯克利分校）；Tsinghua University（清华大学）；Singapore-MIT Alliance for Research and Technology（新加坡-麻省理工学院研究与技术联盟）
