arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.17250cs.AI

通过输入预测和误位修正加速多模态大语言模型游戏性能

Accelerating Multi-modal LLM Gaming Performance via Input Prediction and Mishit Correction

Ziyang Lin, Zixuan Sun, Sanhorn Chen, Xiaoyang Chen, Roy Zhao

备注：
UIUC 25 Fall CS 498
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

通过输入预测和误位修正方法，显著降低多模态大语言模型游戏性能的推理延迟，提升整体控制效果。

摘要：

实时序列控制代理通常受到推理延迟的瓶颈。即使是适度的每步规划延迟也会破坏控制并降低整体性能。我们提出了一种推测与修正框架，该框架借鉴了推测执行的预测-验证哲学，应用于基于模型的控制中，使用TD-MPC2。在每一步中，一个预训练的世界模型和潜在空间MPC规划器生成一个短时间跨度的动作队列以及预测的潜在轨迹，使代理能够执行多个计划动作而无需立即重新规划。当新观察到达时，系统会测量编码的真实潜在状态与队列中预测的潜在状态之间的不匹配程度。对于小到中等程度的不匹配，一个轻量级的学得修正器对推测动作应用残差更新，该更新是从离线重新规划教师中提炼的。对于大不匹配，代理会安全回退到完全重新规划并清除过期的动作队列。我们研究了两种修正器：一种是门控双塔MLP修正器，另一种是时间Transformer修正器，以解决局部错误和系统性漂移。在DMC Humanoid-Walk任务中的实验表明，我们的方法将规划推理次数从500次减少到282次，提高了端到端的步长延迟25个百分点，并且仅以7.1个百分点的回报减少维持了强大的控制性能。消融结果表明，没有修正的推测执行在更长的范围内不可靠，突显了对不匹配意识的修正对于鲁棒延迟减少的必要性。

英文摘要：

Real-time sequential control agents are often bottlenecked by inference latency. Even modest per-step planning delays can destabilize control and degrade overall performance. We propose a speculation-and-correction framework that adapts the predict-then-verify philosophy of speculative execution to model-based control with TD-MPC2. At each step, a pretrained world model and latent-space MPC planner generate a short-horizon action queue together with predicted latent rollouts, allowing the agent to execute multiple planned actions without immediate replanning. When a new observation arrives, the system measures the mismatch between the encoded real latent state and the queued predicted latent. For small to moderate mismatch, a lightweight learned corrector applies a residual update to the speculative action, distilled offline from a replanning teacher. For large mismatch, the agent safely falls back to full replanning and clears stale action queues. We study both a gated two-tower MLP corrector and a temporal Transformer corrector to address local errors and systematic drift. Experiments on the DMC Humanoid-Walk task show that our method reduces the number of planning inferences from 500 to 282, improves end-to-end step latency by 25 percent, and maintains strong control performance with only a 7.1 percent return reduction. Ablation results demonstrate that speculative execution without correction is unreliable over longer horizons, highlighting the necessity of mismatch-aware correction for robust latency reduction.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑