arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.15768cs.AIcs.CY
收藏

ALSO: 用于社交代理的对抗在线策略优化

ALSO: Adversarial Online Strategy Optimization for Social Agents

School of Artificial Intelligence, Tianjin University, Tianjin, China(天津大学人工智能学院,天津,中国);The Chinese University of Hong Kong, Shenzhen, China(香港中文大学(深圳));East China Normal University, Shanghai, China(华东师范大学,上海,中国)

Xiang Li, Liping Yi, Mingze Kong, Min Zhang, Zhongxiang Dai, QingHua Hu

AI总结:

本文提出ALSO框架,通过将多轮交互建模为对抗性带薪问题,并引入轻量级神经代理来预测奖励,从而在动态环境中实现社交代理的鲁棒策略优化。

摘要:

社交模拟为研究社会智能提供了一个有力的测试平台,其中代理在不断变化的上下文中通过多轮对话进行交互并战略性地适应对手。此类环境本质上是非平稳的,要求代理动态调整其策略。然而,大多数基于大型语言模型(LLM)的社会代理依赖于静态人设,而现有的增强社会智能的方法,如离线强化学习或外部规划器,不适用于这些设置,通常假设平稳性并导致显著的训练开销。为弥合这一差距,我们提出了ALSO(对抗性在线策略优化),这是首个用于多代理社交模拟的在线策略优化框架。ALSO通过两个关键贡献提升了社会适应性:(1)ALSO将多轮交互建模为对抗性带薪问题,其中静态人设和动态策略指令的组合被视为臂,提供了一种不依赖环境稳定性假设的解决方案;(2)为了预测奖励并泛化多轮对话中的稀疏反馈,ALSO引入了轻量级神经代理来从交互历史中预测奖励,从而实现高效样本探索和持续在线适应。在Sotopia基准测试中,ALSO在动态环境中一致优于静态基线和现有优化方法,验证了对抗性在线策略优化在构建鲁棒社会代理方面的有效性。

英文摘要:

Social simulation provides a compelling testbed for studying social intelligence, where agents interact through multi-turn dialogues under evolving contexts and strategically adapting opponents. Such environments are inherently non-stationary, requiring agents to dynamically adjust their strategies over time. However, most Large Language Model (LLM) based social agents rely on static personas, while existing approaches for enhancing social intelligence, such as offline reinforcement learning or external planners, are ill-suited to these settings, typically assuming stationarity and incurring substantial training overhead. To bridge this gap, we propose \textbf{ALSO} (\textbf{A}dversarial on\textbf{L}ine \textbf{S}trategy \textbf{O}ptimization), the first framework for online strategy optimization in multi-agent social simulation. ALSO advances social adaptation through two key contributions. (1) ALSO formulates multi-turn interaction as an adversarial bandit problem, where combinations of static personas and dynamic strategy instructions are treated as arms, providing a principled solution to non-stationarity without relying on environmental stability assumptions. (2) To predict rewards and generalize sparse feedback in multi-turn dialogues, ALSO introduces a lightweight neural surrogate to predict rewards from interaction histories, enabling sample-efficient exploration and continuous online adaptation. Experiments on the Sotopia benchmark demonstrate that ALSO consistently outperforms static baselines and existing optimization methods in dynamic environments, validating the effectiveness of adversarial online strategy optimization for building robust social agents.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑