探索基于文本的因果混淆调整中的稀疏自编码器
Exploring Sparse Autoencoders in Text-Based Causal Confounding Adjustment
浏览论文内容
中文总结 AI 辅助
该研究针对文本因果混淆调整的表示权衡问题，提出基于稀疏自编码器（SAE）的迭代特征选择流程，经半合成评估验证其调整效果更优，还针对多标签未观测混淆变量的复杂场景开展了评估。
中文摘要 AI 辅助
在许多场景中，基于文本数据研究因果问题需要调整文本内部的混淆信息。然而，构建用于调整的文本表示存在一个权衡：它们必须足够大且/或密集，以保留无偏效应估计所需的混淆变量；同时又必须足够小且/或稀疏，以满足有限样本重叠并产生低方差估计。为解决这一权衡，我们转向稀疏自编码器（SAEs），并提出一种新颖的因果调整流程，该流程通过条件独立性测试迭代选择最小的SAE特征集。我们在具有二元混淆变量的标准半合成评估中发现，SAE表示比其他替代表示实现了更好的调整（更低的偏差和更高的覆盖率），且其可解释性为证伪提供了机会。我们还引入了更具现实性的半合成评估，该评估使用多标签数据作为未观测到的混淆变量，发现现成的调整方法需要针对这些更复杂的设置进行更多研究。代码：this https URL
英文摘要
In many settings, studying causal questions based on text data requires adjusting for confounding information within texts. Yet there is a tradeoff in constructing text representations for adjustment: they must be sufficiently large and/or dense to preserve the confounding variables necessary for unbiased effect estimation, but sufficiently small and/or sparse to satisfy finite-sample overlap and yield low-variance estimates. To address this tradeoff, we turn to sparse autoencoders (SAEs), and propose a novel causal adjustment pipeline that iteratively selects a minimal set of SAE features via conditional independence tests. We find that SAE representations achieve better adjustments (lower bias and and higher coverage) than alternative representations in standard semi-synthetic evaluations with binary confounders, and their interpretability offers opportunities for falsification. We also introduce a more realistic semi-synthetic evaluation that uses multi-label data as the unobserved confounders and find off-the-shelf adjustment methods require increased investigation for these more complex settings. Code: https://github.com/mianzg/sae-text-confounder
发表机构
- Johns Hopkins University（约翰斯·霍普金斯大学）
- Williams College（威廉姆斯学院）
- Cohere
机构由 AI 辅助整理，请以论文原文为准。
