D模型和E模型：大语言模型采样行为中的多样性-稳定性权衡
D-Models and E-Models: Diversity-Stability Trade-offs in the Sampling Behavior of Large Language Models
AI总结：
研究揭示了大语言模型中D模型与E模型在采样行为上的多样性-稳定性权衡，为任务应用中的模型选择提供了指导。
摘要：
大语言模型（LLMs）中下一个标记的预测概率（P_token）与下一个信息的相关性概率、下一个产品的购买概率以及下一个动作的执行概率紧密相关，这些都属于任务级目标分布（P_task）的范围。尽管LLMs已知能够生成近似现实世界分布的样本，但其细粒度采样概率是否忠实反映任务需求仍是一个开放性问题。通过受控的分布-采样模拟，我们揭示了LLM行为中存在明显的二元对立，区分出两种模型类型：D模型（例如Qwen-2.5），其P_token表现出较大的步间变化且与P_task对齐较差；而E模型（例如Mistral-Small），其P_token更为稳定且更符合P_task。我们进一步在代码生成和推荐等下游任务中评估这两种模型类型，揭示了多样性与稳定性之间的系统性权衡，从而影响任务结果。最后，我们分析这两种模型家族的内部特性，以探究其底层机制。这些发现为LLM的概率采样行为提供了基础见解，并为何时选择D模型或E模型提供了实际指导。对于大规模网络应用，包括推荐、搜索和对话代理，我们的结果为模型选择和配置提供了依据，以在现实世界不确定性下平衡多样性与可靠性，提供更高级的解释。
英文摘要：
The predictive probability of the next token (P_token) in large language models (LLMs) is inextricably linked to the probability of relevance for the next piece of information, the purchase probability of the next product, and the execution probability of the next action-all of which fall under the scope of the task-level target distribution (P_task). While LLMs are known to generate samples that approximate real-world distributions, whether their fine-grained sampling probabilities faithfully align with task requirements remains an open question. Through controlled distribution-sampling simulations, we uncover a striking dichotomy in LLM behavior, distinguishing two model types: D-models (e.g. Qwen-2.5), whose P_token exhibits large step-to-step variability and poor alignment with P_task; and E-models (e.g. Mistral-Small), whose P_token is more stable and better aligned with P_task. We further evaluate these two model types in downstream tasks such as code generation and recommendation, revealing systematic trade-offs between diversity and stability that shape task outcomes. Finally, we analyze the internal properties of both model families to probe their underlying mechanisms. These findings offer foundational insights into the probabilistic sampling behavior of LLMs and provide practical guidance on when to favor D- versus E-models. For web-scale applications, including recommendation, search, and conversational agents, our results inform model selection and configuration to balance diversity with reliability under real-world uncertainty, providing a better level of interpretation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
