AI 中文总结
本文提出基于能量的上下文内密度估计模型ICED，其为Transformer架构，预训练于合成先验，可单次前向传播返回未归一化对数密度，无需重训练调优，可驱动四项任务且性能比肩任务特定最强方法。
AI 中文摘要
密度估计是表格数据上诸多无监督任务的基础，例如异常检测、分布外检测和数据增强。尽管所有这些问题都可归结为关于概率质量所在位置的问题，但它们通常是通过为每个数据集单独拟合一个模型来分别解决的，每个模型都有自己的超参数和调优预算。我们推出ICED，一种基于能量的上下文内密度估计器，它消除了每个数据集的成本。ICED是一个基于Transformer的模型，针对密度估计预训练于一个专门构建的合成先验之上，其目标是在信息丰富的地方拟合对数密度，并在其他地方保持其排序。在推理过程中，它将一个数据集作为上下文读取，并在一次前向传播中为任何查询点返回未归一化的对数密度，无需拟合、采样或超参数选择。单个冻结的ICED模型可驱动四项通常由四个专用流程处理的任务：密度估计、分布外检测、无监督异常检测和生成增强。在所有四项任务中，它都与最强的任务特定方法具有竞争力，同时是唯一一种在任务间切换时无需重新训练、无需调优且无需标签的方法。代码可在该https URL获取。
英文摘要
Density estimation underlies many unsupervised tasks on tabular data such as anomaly detection, out-of-distribution detection, and data augmentation. Although all these problems reduce to questions about where probability mass lies, they are typically solved individually by fitting a separate model to each dataset, with its own hyperparameters and tuning budget. We introduce ICED, an in-context, energy-based density estimator that removes this per-dataset cost. ICED is a transformer-based model pretrained once on a synthetic prior built specifically for density estimation under an objective that fits log-density where it is informative and preserves its ordering elsewhere. In the inference, it reads a dataset as context and returns an unnormalized log-density for any query point in a single forward pass, with no fitting, sampling, or hyperparameter selection. A single frozen ICED model then drives four tasks usually handled by four specialized pipelines: density estimation, out-of-distribution detection, unsupervised anomaly detection, and generative augmentation. Across all four, it is competitive with the strongest task-specific method, while being the only approach that needs no retraining, no tuning, and no labels to move between them. The code is available at https://github.com/gmum/iced.
