arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.05545stat.MEecon.EM
收藏

语言模型能否在无统计偏见的情况下提升随机实验的效力?

Can Language Models Boost the Power of Randomized Experiments Without Statistical Bias?

University of California, Berkeley(加州大学伯克利分校);University of California, San Diego(加州大学圣迭戈分校);Michael G. Foster School of Business, University of Washington(华盛顿大学迈克尔G福斯特商学院);University of Southern California(南加州大学)

Xinrui Ruan, Xinwei Ma, Yingfei Wang, Waverly Wei, Jingshen Wang

AI总结:

CALM通过整合语言模型生成的洞察与因果估计器,提升随机实验的精度和有效性,减少偏见并提高稳定性。

摘要:

随机实验或随机对照试验(RCT)是因果推断的黄金标准,但成本和样本量限制了其效力。我们引入了CALM(利用语言模型进行因果分析)统计框架,该框架整合了大型语言模型(LLM)生成的RCT洞察与已建立的因果估计器,以提高精度同时保持统计有效性。具体而言,CALM将LLM生成的输出视为辅助预测信息,并通过异质校准步骤校正其潜在的偏见,该步骤通过残差化和最优重新加权预测来实现。我们证明,即使LLM预测存在偏见,CALM仍保持一致性,并在各种因果效应上比增强逆概率加权估计器更有效。特别地,CALM开发了一种少量样本变体,该变体在随机采样的演示集上聚合预测。所得到的U统计量似预测恢复了独立同分布结构,并且也缓解了提示选择的变异性。在模拟中,CALM在经过校准的移动应用抑郁RCT中表现出比其他基准方法更低的方差,且在零样本和少量样本设置中有效,并且在不同提示设计下保持稳定。通过系统利用LLM来利用非结构化数据和预训练期间学到的外部知识,CALM为RCT中的更精确因果分析提供了一条实用路径。

英文摘要:

Randomized controlled trials (RCTs) are widely adopted for causal inference, yet cost and sample-size constraints limit power. We introduce CALM (Causal Analysis leveraging Language Models), a statistical framework that integrates insights generated by large language models (LLMs) into the analysis of RCTs using established causal estimators to increase precision while preserving statistical validity. In particular, CALM treats LLM-generated outputs as auxiliary prognostic information and corrects their potential bias via a heterogeneous calibration step that residualizes and optimally reweights predictions. We prove that CALM remains consistent even when LLM predictions are biased and achieves efficiency gains over augmented inverse probability weighting estimators for various causal estimands. In particular, CALM develops a few-shot variant that aggregates predictions across randomly sampled demonstration sets. The resulting U-statistic-like predictor restores i.i.d. structure and also mitigates prompt-selection variability. Empirically, in simulations calibrated to a mobile-app depression RCT, CALM delivers lower variance relative to other benchmarking methods, is effective in zero- and few-shot settings, and remains stable across prompt designs. By principled use of LLMs to harness unstructured data and external knowledge learned during pretraining, CALM provides a practical path to more precise causal analyses.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑