arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.01468cs.CLcs.LG
收藏

记住你做了什么,以便知道下一步该做什么

Remember what you did so you know what to do next

Information Sciences Institute, University of Southern California(信息科学研究所,南加州大学)

Manuel R. Ciosici, Alex Hedges, Yash Kankanampati, Justin Martin, Marjorie Freedman, Ralph Weischedel

AI总结:

本研究使用中等规模的大语言模型GPT-J为模拟机器人在ScienceWorld文本游戏中的30类目标生成计划,通过填充更多历史步骤显著提升性能,并发现任务平均可能掩盖性能差异。

摘要:

我们探索使用中等规模的大语言模型(GPT-J,6B参数)为模拟机器人在ScienceWorld(一个用于基础科学实验的文本游戏模拟器)中制定计划,以实现30类目标。先前发表的实证工作声称,与强化学习相比,大语言模型(LLMs)不太适合(Wang等人,2022)。使用马尔可夫假设(仅前一步),LLM的性能是强化学习方法性能的1.4倍。当我们尽可能多地填充LLM的输入缓冲区(包含尽可能多的先前步骤)时,性能提升至3.5倍。即使仅使用6.5%的训练数据,我们观察到性能比强化学习方法提高了2.2倍。我们的实验表明,30类动作的性能差异很大,这表明对任务进行平均可能会掩盖显著的性能问题。在与我们同时期的工作中,Lin等人(2023)展示了一种两部分方法(SwiftSage),该方法使用一个小型LLM(T5-large)并辅以OpenAI的大规模LLM,在ScienceWorld中取得了出色结果。我们的6B参数单阶段GPT-J在结合GPT-3.5 turbo(其参数数量是GPT-J的29倍)时,与SwiftSage的两阶段架构性能相匹配。

英文摘要:

We explore using a moderately sized large language model (GPT-J 6B parameters) to create a plan for a simulated robot to achieve 30 classes of goals in ScienceWorld, a text game simulator for elementary science experiments. Previously published empirical work claimed that large language models (LLMs) are a poor fit (Wang et al., 2022) compared to reinforcement learning. Using the Markov assumption (a single previous step), the LLM outperforms the reinforcement learning-based approach by a factor of 1.4. When we fill the LLM's input buffer with as many prior steps as possible, improvement rises to 3.5x. Even when training on only 6.5% of the training data, we observe a 2.2x improvement over the reinforcement-learning-based approach. Our experiments show that performance varies widely across the 30 classes of actions, indicating that averaging over tasks can hide significant performance issues. In work contemporaneous with ours, Lin et al. (2023) demonstrated a two-part approach (SwiftSage) that uses a small LLM (T5-large) complemented by OpenAI's massive LLMs to achieve outstanding results in ScienceWorld. Our 6-B parameter, single-stage GPT-J matches the performance of SwiftSage's two-stage architecture when it incorporates GPT-3.5 turbo which has 29-times more parameters than GPT-J.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑