Recent advancements on Large Language Models (LLMs) enable AI Agents to automatically generate and execute multi-step plans to solve complex tasks. However, since LLM's content generation process is hardly controllable, current LLM-based agents frequently generate invalid or non-executable plans, which jeopardizes the performance of the generated plans and corrupts users' trust in LLM-based agents. In response, this paper proposes a novel "Formal-LLM" framework for LLM-based agents by integrating the expressiveness of natural language and the precision of formal language. Specifically, the framework allows agent developers to express their requirements or constraints for the planning process as an automaton. A stack-based LLM plan generation process is then conducted under the supervision of the automaton to ensure that the generated plan satisfies the constraints, making the planning process controllable. We conduct experiments on both benchmark tasks and practical real-life tasks, and our framework achieves over 50% overall performance increase, which validates the feasibility and effectiveness of employing Formal-LLM to guide the plan generation of agents, preventing the agents from generating invalid and unsuccessful plans. Further, more controllable LLM-based agents can facilitate the broader utilization of LLM in application scenarios where high validity of planning is essential. The source code of this work is available at https://github.com/agiresearch/Formal-LLM.
ReMe: Scaffolding Personalized Cognitive Training via Controllable LLM-Mediated Conversations
Microsoft Research（微软研究院）；Department of Geriatric Psychiatry, Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属上海市精神卫生中心老年精神科）
Graph Your Way to Inspiration: Integrating Co-Author Graphs with Retrieval-Augmented Generation for Large Language Model Based Scientific Idea Generation
School of Computing Newcastle University（计算学院新castle大学）
DVM: Towards Controllable LLM Agents in Social Deduction Games
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Singapore Management University（新加坡管理大学）；University of Science and Technology of China（中国科学技术大学）
HACHIMI: Scalable and Controllable Student Persona Generation via Orchestrated Agents
East China Normal University（华东师范大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；Shanghai Innovation Institute（上海创新研究院）
SciR: A Controllable Benchmark for Scientific Reasoning in LLMs
Idiap Research Institute（Idiap研究 institute）；EPFL（瑞士联邦理工学院）；School of Computer Science, University of Sheffield（谢菲尔德大学计算机科学学院）；University of Manchester（曼彻斯特大学）；National Biomarker Centre, CRUK Manchester Institute（国家生物标志物中心，CRUK曼彻斯特研究所）
ControlTraj: Controllable Trajectory Generation with Topology-Constrained Diffusion Model
Southern University of Science and Technology（南方科技大学）；City University of Hong Kong（香港城市大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；York University（约克大学）
Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings
Google Research（谷歌研究院）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
风格转换是将句子改写成目标风格，同时大致保留内容的任务。虽然大多数先前的文献假设访问大型风格标记语料库，但最近的工作（Riley et al. 2021）尝试了“Few-Shot”风格迁移，仅使用3-10个句子进行风格提取的推理。在这项工作中，我们研究了一个相关的低资源设置：没有风格标记语料库的语言的风格转移。我们注意到，现有的Few-Shot方法执行此任务很差，通常逐字复制输入。我们推动了Few-Shot风格转换的最新技术，提出了一种新的方法来模拟释义之间的风格差异。与以前的工作相比，我们的模型在七种语言的正式传输和代码混合添加方面的性能提高了2- 3倍。此外，我们的方法是更好地控制风格转移幅度使用输入标量旋钮。我们报告了几个属性转移任务（情感转移，简化，性别中立化，文本匿名化）的有希望的定性结果，所有这些都没有重新训练模型。最后，我们发现模型评估是困难的，由于缺乏数据集和许多语言的指标。为了方便未来的研究，我们众包正式注释4000句对四种印度语言，并使用这些数据来设计我们的自动评估。
