The popularity of Large Language Models (LLMs) have unleashed a new age ofLanguage Agents for solving a diverse range of tasks. While contemporary frontier LLMs are capable enough to power reasonably good Language agents, the closed-API model makes it hard to improve in cases they perform sub-optimally. To address this, recent works have explored ways to improve their performance using techniques like self-reflection and prompt optimization. Unfortunately, techniques like self-reflection can be used only in an online setup, while contemporary prompt optimization techniques are designed and tested to work on simple tasks. To this end, we introduce MetaReflection, a novel offline reinforcement learning technique that enhances the performance of Language Agents by augmenting a semantic memory based on experiential learnings from past trials. We demonstrate the efficacy of MetaReflection by evaluating across multiple domains, including complex logical reasoning, biomedical semantic similarity, open world question answering, and vulnerability threat detection, in Infrastructure-as-Code, spanning different agent designs. MetaReflection boosts Language agents' performance by 4% to 16.82% over the raw GPT-4 baseline and performs on par with existing state-of-the-art prompt optimization techniques while requiring fewer LLM calls.
TasTe: Teaching Large Language Models to Translate through Self-Reflection
Institute of Computing and Intelligence, Harbin Institute of Technology(哈尔滨工业大学计算与智能学院);Pattern Recognition Center, WeChat AI, Tencent Inc(腾讯微信AI模式识别中心)
ChartPoint: Guiding MLLMs with Grounding Reflection for Chart Reasoning
Tsinghua University(清华大学);International Digital Economy Academy(国际数字经济学院);Beihang University(北航);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction
Beijing University of Posts and Telecommunications(北京邮电大学);AI Lab of Lenovo Research(联想研究院AI实验室);Tianjin University(天津大学);The Hong Kong University of Science and Technology(香港科技大学)
mR$^2$AG: Multimodal Retrieval-Reflection-Augmented Generation for Knowledge-Based VQA
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information(多模态信息超级智能安全北京市重点实验室);Tencent Inc.(腾讯公司);Huawei Noah’s Ark Laboratory(华为诺亚方舟实验室)
Self-Consistency as a Free Lunch: Reducing Hallucinations in Vision-Language Models via Self-Reflection
Department of Computer Vision, Mohamed Bin Zayed University of Artificial Intelligence(计算机视觉系,Mohamed Bin Zayed人工智能大学);School of Information Science and Technology, University of Science and Technology of China(信息科学与技术学院,中国科学技术大学);Bytedance Seed(字节跳动种子);School of Computer Science, Qinghai Normal University(计算机科学学院,青海师范大学)
CMMR-VLN: Vision-and-Language Navigation via Continual Multimodal Memory Retrieval
Foshan Graduate School of Innovation at Northeastern University(东北大学创新研究生院);Faculty of Robot Science and Engineering at Northeastern University(东北大学机器人科学与工程学院);College of Software at Northeastern University(东北大学软件学院);School of Aeronautic Science and Engineering at Beihang University(北航航空科学与工程学院)
A Pragmatist Robot: Learning to Plan Tasks by Experiencing the Real World
Robotic Systems Lab, ETH Zürich(瑞士联邦理工学院机器人系统实验室);ETH AI Center, ETH Zürich(瑞士联邦理工学院人工智能中心);Huawei Noah’s Ark Lab, London, UK(华为诺亚实验室,伦敦,英国);UCL Centre for AI, London, UK(伦敦大学学院人工智能中心)
Advancing Tool-Augmented Large Language Models via Meta-Verification and Reflection Learning
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(认知智能国家重点实验室,中国科学技术大学);Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology(社会计算与交互机器人研究中心,哈尔滨工业大学)
PSPA-Bench: A Personalized Benchmark for Smartphone GUI Agent
Northwestern Polytechnical University(西北工业大学);Tsinghua University(清华大学);Peking University(北京大学);Beijing Institute of Mathematical Sciences and Applications(北京数学科学与应用研究院)
RoT: Enhancing Large Language Models with Reflection on Search Trees
School of Information Science and Technology, ShanghaiTech(上海科技大学信息科学与技术学院);Shanghai Engineering Research Center of Intelligent Vision and Imaging(上海智能视觉与成像工程技术研究中心)
MobileSteward: Integrating Multiple App-Oriented Agents with Self-Evolution to Automate Cross-App Instructions
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Xiaomi AI Lab(小米AI实验室);School of Computer Science, Wuhan University(武汉大学计算机学院)