发表机构
UNSW Sydney; Harz University of Applied Sciences（新南威尔士大学; 哈尔茨应用科学大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文研究CEM中世界模型评分误差对决策和候选生成的双重影响，通过四种模型和十二个单元实验，发现评分器差异大于候选池差异，并验证了环境排序更新可降低最终成本。
AI 中文摘要
交叉熵方法（CEM）利用世界模型评分来选择动作序列，并拟合其下一次迭代中采样的分布。因此，评分误差既可能改变当前决策，也可能改变后续考虑的候选方案。我们分别评估了这两个角色。四种预测模型生成CEM轨迹，每个模型对每个保存的候选池进行重新评分。在环境中执行相同的候选方案，提供了一个参考精英集和提案更新。在Walker和Cheetah上的十二个独立训练的任务-种子单元中，预设的提案距离从第一次到最后一次CEM迭代在每个单元中均有所下降。提案宽度收缩，拟合均值相对于剩余搜索宽度分离。在Walker上，成对排序一致性接近随机水平，在Cheetah上则下降；精英集一致性没有改善。这一比较显示，在Cheetah上，评分器之间的差异大于候选池来源之间的差异；Walker在两者及其配对中均有差异。我们使用原始的六个单元选择随机非线性进行单次更新干预，而不检查干预结果。用环境排序的更新替换其首次模型排序的更新，在这六个单元以及从选择中保留的另外六个单元中，降低了最终实现的选择序列成本。
英文摘要
The cross-entropy method (CEM) uses world-model scores to select action sequences and fit the distribution sampled in its next iteration. A scoring error can therefore change both the present decision and the candidates considered later. We evaluate these two roles separately. Four types of predictive model generate CEM traces, and every model rescores every saved candidate pool. Executing the same candidates in the environment provides a reference elite set and proposal update. Across twelve independently trained task-seed units on Walker and Cheetah, the pre-specified proposal distance falls from the first to the final CEM iteration in every unit. Proposal widths contract and fitted means separate relative to the remaining search width. Pairwise ranking agreement stays near chance on Walker and declines on Cheetah; elite-set agreement does not improve. This comparison shows greater variation between scorers than between pool sources on Cheetah; Walker has variation in both and in their pairings. We use the original six units to select Random nonlinear for a one-update intervention, without inspecting intervention outcomes. Replacing its first model-ranked update with an environment-ranked update lowers final realised selected-sequence cost in those six units and in six further units held out from the selection.
Comments20 pages, including 12 pages appendix
