arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.07983cs.AIcs.CL
收藏

通过代理假设和实验加速社会科学研究

Accelerating Social Science Research via Agentic Hypothesization and Experimentation

Adobe Media and Data Science Research (MDSR)(Adobe媒体与数据科学研究所)

Jishu Sen Gupta, Harini SI, Somesh Kumar Singh, Syed Mohamad Tawseeq, Yaman Kumar Singla, David Doermann, Rajiv Ratn Shah, Balaji Krishnamurthy

AI总结:

EXPERIGEN通过代理框架实现端到端的科学发现,发现更多显著且预测性强的假设,并通过A/B测试验证其有效性。

摘要:

数据驱动的社会科学研究本质上是缓慢的,依赖于观察、假设生成和实验验证的迭代循环。尽管最近的数据驱动方法承诺加速这一过程的一部分,但它们大多未能支持端到端的科学发现。为了解决这一差距,我们引入了EXPERIGEN,一个代理框架,通过受贝叶斯优化启发的两阶段搜索,将端到端的发现过程具体化,其中生成器提出候选假设,实验者则通过实证方法评估它们。在多个领域中,EXPERIGEN始终发现比先前方法多2-4倍的统计上显著的假设,且这些假设的预测性比以往方法高出7-17个百分点,并自然扩展到复杂的数据领域,包括多模态和关系数据集。除了统计性能之外,假设必须是新颖的、实证的,并且具有行动性,以推动真正的科学进步。为了评估这些品质,我们对机器生成的假设进行了专家审查,收集了来自高级教职员工的反馈。在25个审查的假设中,88%被评定为中等或强烈新颖,70%被认为具有影响性并值得追求,大多数展示了与高级研究生级研究相当的严谨性。最后,认识到最终验证需要现实世界的证据,我们进行了第一个LLM生成假设的A/B测试,观察到统计上显著的结果,p值小于1e-6,且效应量为344%。

英文摘要:

Data-driven social science research is inherently slow, relying on iterative cycles of observation, hypothesis generation, and experimental validation. While recent data-driven methods promise to accelerate parts of this process, they largely fail to support end-to-end scientific discovery. To address this gap, we introduce EXPERIGEN, an agentic framework that operationalizes end-to-end discovery through a Bayesian optimization inspired two-phase search, in which a Generator proposes candidate hypotheses and an Experimenter evaluates them empirically. Across multiple domains, EXPERIGEN consistently discovers 2-4x more statistically significant hypotheses that are 7-17 percent more predictive than prior approaches, and naturally extends to complex data regimes including multimodal and relational datasets. Beyond statistical performance, hypotheses must be novel, empirically grounded, and actionable to drive real scientific progress. To evaluate these qualities, we conduct an expert review of machine-generated hypotheses, collecting feedback from senior faculty. Among 25 reviewed hypotheses, 88 percent were rated moderately or strongly novel, 70 percent were deemed impactful and worth pursuing, and most demonstrated rigor comparable to senior graduate-level research. Finally, recognizing that ultimate validation requires real-world evidence, we conduct the first A/B test of LLM-generated hypotheses, observing statistically significant results with p less than 1e-6 and a large effect size of 344 percent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑