大语言模型使用与科学生产力之间的稳健关联：评估停止时间选择偏差
A robust association between LLM use and scientific productivity: Assessing stopping-time selection
浏览论文内容
中文总结 AI 辅助
本研究针对RBB提出的LLM采用时间定义引发的停止时间选择偏差，通过重新校准安慰剂检验及多种互补设计验证，证实该偏差规模有限，LLM使用与科学生产力的正向关联稳健存在。
中文摘要 AI 辅助
Renault、Bergeaud和Bosquet（以下简称RBB）提出，将大语言模型（LLM）采用时间定义为作者摘要首次被标记的月份，会引发停止时间选择偏差，即便不存在因果效应，也可能产生正向的事件研究路径。尽管这一机制在数学上成立，但它并不能证明零效应。我们将RBB自身的随机安慰剂检验重新校准至检测器的实际标记率，结果显示测得的关联度远高于这一基准，因此该人为偏差的规模太小，不足以解释生产力变化。我们进一步通过一系列不会受时间偏差影响估计的互补设计，重新评估了LLM采用与生产力之间的关联：以某一年为采用时间、另一年为产出测量期的前后对比法、双重差分法的保守对照组、从未定义采用日期的基于强度的模型设定，以及保持标记率固定的基于秩的测量方法。所有这些估计结果都显示出正向的生产力关联，而在ChatGPT出现前的安慰剂数据上运行相同检验则得到零效应。RBB指出的偏差确实存在，但规模有限，无法解释我们报告的研究模式。
英文摘要
Renault, Bergeaud, and Bosquet (hereafter RBB) argue that dating LLM adoption as the first month in which an author's abstract is flagged induces a stopping-time selection that can produce a positive event-study path even when there is no causal effect. Although this mechanism is mathematically possible, it does not constitute proof of a null effect. Recalibrating RBB's own random placebo to the detector's realized flag rate, we show that the measured association stays well above this benchmark, so the artifact is too small to explain the productivity changes. We further re-estimate the association between LLM adoption and productivity with a series of complementary designs in which the timing artifact cannot bias the estimate: a before-and-after comparison that dates adoption in one year and measures output in another, a conservative control group for difference-in-differences, an intensity-based specification that never defines an adoption date, and a rank-based measurement holding the flag rate fixed. A positive productivity association persists across all of these estimates, while the same tests run on pre-ChatGPT placebo data return null effects. The artifact RBB identify is real but bounded, and it does not account for the pattern we report.
