AI 中文总结
研究大语言模型输出随机性问题及影响，指出其输出因多种因素在重复请求时可变，难以精确再现。通过公司备案文件情感分类说明问题，提出文章、复制包报告标准及对数据编辑和作者的指导，建议将LLM输出视为分布抽样。
AI 中文摘要
大语言模型（LLMs）越来越多地用于为研究生成数据，典型用例包括分类、注释、信息提取和数值分数生成。与传统测量不同，即使提示和明显的模型设置不变，LLM输出在重复请求时也可能不同，这种变化源于故意采样、静默模型更新、数值舍入或专家路由。设置专用温度参数为零可消除故意采样，但无法消除其他随机源。使用专有应用程序编程接口时通常无法精确再现。本地执行开放权重模型可提供更大控制，但可重复性仍取决于完整的硬件和软件堆栈。我们通过公司备案文件的情感分类来说明这些问题，并研究其对下游回归结果的影响。然后，我们为文章和复制包提出了报告标准，以及为数据编辑和作者提供了指导。这些发现和建议共同表明，LLM输出应被视为从分布中抽取的样本，而不是固定的测量值。
英文摘要
Large language models (LLMs) are increasingly used to generate data for research. Typical use cases are classifications, annotations, information extraction, and generation of numerical scores. Unlike conventional measurements, LLM outputs can vary across repeated requests even when the prompt and apparent model settings remain unchanged. This variation arises from deliberate sampling, silent model updates, numerical rounding, or expert routing. Setting a dedicated temperature parameter to zero removes deliberate sampling when that option is available, but it does not eliminate the other sources of randomness. Exact reproduction is therefore generally not possible when using proprietary application programming interfaces. Local execution of open-weight models offers greater control, but reproducibility still depends on the complete hardware and software stack. We illustrate these issues through sentiment classifications of corporate filings and examine their consequences for downstream regression results. We then propose a reporting standard for articles and replication packages, as well as guidance for data editors and authors. Together, these findings and recommendations establish that LLM outputs should be treated as draws from a distribution rather than as fixed measurements.
