证据对齐的离散专家局部组合用于序列恢复
Evidence-Aligned Local Composition of Discrete Experts for Sequence Restoration
- Tufts University（塔夫茨大学）
- Boston College（波士顿学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
提出证据对齐的局部专家组合方法，通过去噪损失推断逐位置软权重，在无标签和路由器下恢复混合文档，字段准确率达0.85-0.98。
AI中文摘要：
一个被建模为离散令牌序列的文档可以被认为是来自不同领域的文本组合生成的；例如，一个README文件会在散文、代码和配置之间切换。当这样的文档被损坏且只有冻结的领域专家可用时，恢复它需要在测试时且没有区域标签或训练好的路由器的情况下，决定每个位置缺失了什么以及该信任哪个专家。我们引入了证据对齐的局部组合，该方法在给定的损坏模型下，根据损坏观测的边缘证据推断出专家上的软性、逐位置的权重，通过专家自身的去噪损失估计证据，并平滑跨位置的权重。由于权重是软的，当真实组合是混合时，它恢复出混合体，而当单一专家足够时则集中在该专家上。在分类模拟器、字节级专家以及从1.3B离散流匹配模型微调的专家上，推断的权重在自然混合的科学文档上以0.85的字段准确度跟踪真实区域，在区域词汇不重叠的构造混合体上以0.98的准确度跟踪。当专家真正不同时，恢复效果优于单一的全局权重，而当专家趋同时则退化为该权重，跟踪专家分离度的度量。
英文摘要：
A document modeled as a discrete sequence of tokens can be thought of as being generated from a composition of texts from different domains; a README file, for example, moves between prose, code, and configuration. When such a document is corrupted and only frozen domain experts are available, restoring it requires deciding both what is missing and which expert to trust at each position, at test time and without region labels or a trained router. We introduce evidence-aligned local composition, which infers a soft, position-wise weighting over the experts from the marginal evidence of the corrupted observation under a given corruption model, estimating the evidence from the experts' own denoising losses and smoothing the weights across positions. Because the weighting is soft, it recovers a mixture when the true composition is mixed and concentrates on one expert when that suffices. Across a categorical simulator, byte-level experts, and experts fine-tuned from a $1.3$B discrete flow-matching model, the inferred weights track the true regions at $0.85$ field accuracy on naturally mixed scientific documents, and at $0.98$ on constructed mixtures whose regions are lexically disjoint. Restoration improves over a single global weight when the experts are genuinely distinct and reduces to it when they converge, tracking a measure of expert separation.
