arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.06087cs.CLcs.AIcs.LG
收藏

通过语言模型机制构建决策模型

Building Decision Making Models Through Language Model Regime

Alibaba Group(阿里巴巴集团)

Yu Zhang, Haoxiang Liu, Feijun Jiang, Weihua Luo, Kaifu Zhang

AI总结:

本文提出LTU两阶段方法,通过跨领域预训练构建基础决策模型并面向场景微调,利用LLM增强单步与多步决策的泛化能力。

摘要:

我们提出了一种利用大型语言模型(LLM)泛化能力解决决策问题的新方法。专家系统、规划算法和强化学习等传统方法往往表现出有限的泛化能力,通常需要为每个独特任务训练新模型。相比之下,LLM在跨各类语言任务的泛化方面取得了显著成功,这启发了一种训练决策模型的新策略。我们的方法称为“Learning then Using”(LTU),包含两个阶段。首先,learning(学习)阶段通过整合来自不同领域和决策情境的多样知识,开发一个稳健的基础决策模型。随后的using(使用)阶段针对特定决策场景微调该基础模型。与其他通过监督学习利用LLM进行决策的研究不同,我们的LTU方法采用一种多用途训练方法,将广泛预训练与有针对性的微调相结合。在广告和搜索优化等电子商务领域的实验表明,LTU方法在决策能力和泛化能力方面优于传统监督学习机制。LTU方法是首个结合LLM、同时适用于单步和多步决策任务的实用训练架构,可应用于游戏和机器人领域之外。它为决策提供了稳健且可适应的框架,提升了各类系统应对多种挑战的有效性和灵活性。

英文摘要:

We propose a novel approach for decision making problems leveraging the generalization capabilities of large language models (LLMs). Traditional methods such as expert systems, planning algorithms, and reinforcement learning often exhibit limited generalization, typically requiring the training of new models for each unique task. In contrast, LLMs demonstrate remarkable success in generalizing across varied language tasks, inspiring a new strategy for training decision making models. Our approach, referred to as "Learning then Using" (LTU), entails a two-stage process. Initially, the \textit{learning} phase develops a robust foundational decision making model by integrating diverse knowledge from various domains and decision making contexts. The subsequent \textit{using} phase refines this foundation model for specific decision making scenarios. Distinct from other studies that employ LLMs for decision making through supervised learning, our LTU method embraces a versatile training methodology that combines broad pre-training with targeted fine-tuning. Experiments in e-commerce domains such as advertising and search optimization have shown that LTU approach outperforms traditional supervised learning regimes in decision making capabilities and generalization. The LTU approach is the first practical training architecture for both single-step and multi-step decision making tasks combined with LLMs, which can be applied beyond game and robot domains. It provides a robust and adaptable framework for decision making, enhances the effectiveness and flexibility of various systems in tackling various challenges.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑