AI 中文总结
提出DAST方法，通过重排token顺序结合算术采样，在不改变分布且开销极小的情况下提高LLM输出多样性，并显著提升ProtoQA任务性能。
AI 中文摘要
我们提出了DAST（使用TokenTour的多样化算术采样），一种在不改变边际分布且生成时间开销可忽略（几微秒）的情况下提高LLM输出多样性的方法。我们观察到，token ID通常以无意义的顺序排列，我们重新分配它们，使具有相似含义的token连续出现。这可以在每个模型上提前几百秒完成，所得顺序可用于所有后续生成。通过将此顺序与算术采样（或准蒙特卡洛方法）结合，我们在保持分布的同时，使相似token在多次运行中生成的可能性降低。我们的方法不仅产生定性好的想法，而且显著提高了下游任务ProtoQA的性能。
英文摘要
We propose DAST (Diversifying Arithmetic Sampling with TokenTour), a method that increases the diversity of LLM outputs without any change to the marginal distribution and with negligible generation-time overhead (a few microseconds). We observe that token IDs are often arranged in a meaningless order and reassign them so that tokens with similar meanings appear consecutively. This can be done in advance in a few hundred seconds per model, and the resulting order can be reused for all subsequent generations. By combining this order with arithmetic sampling (or quasi-Monte Carlo methods), we make similar tokens less likely to be generated across runs while preserving the distribution. Our method not only produces qualitatively good ideas but also significantly improves performance on the downstream task of ProtoQA.
