各向同性却不可解码：潜在预测文本表示中的序列内容充分性缺口

中文摘要 AI 辅助

我们通过研究表示是否保留输入中可用的有序目标信息，来探讨序列内容充分性。一种信息论分解将输入模糊性、表示损失和读出失配分离开来。我们构造了可恢复视图，其中完美一致性和联合各向同性高斯性与零目标信息共存，并确立了确定性规范锚点所施加的限制。令牌对数损失提供了一个单侧信息损失界限；固定惩罚岭回归分析表明，为何仅凭秩无法确定预测风险。这些结果催生了CANOPE，一个具有有序潜在画布、规范令牌监督和几何正则化的非自回归框架。在40,000个验证序列上，在提供正确目标长度且存在强自然损坏的情况下，潜在一致性（PL0）和令牌接地（PL2）的合并秩几乎相同，但分别达到13.5%和98.8%的位置Recall@1。在3,930个LJSpeech验证话语上，冻结的PL2配合训练好的MatchaTTS读出器在损坏文本上达到21.54%的词错误率（WER），而冻结的PL0为99.22%，端到端MatchaTTS则达到10.93%。这些结果表明，在此研究的文本设置中，仅凭几何正则性并不能保证可恢复的序列内容或有效的下游访问。

英文摘要

We study sequential content sufficiency by investigating whether a representation retains the ordered target information available in its input. An information-theoretic decomposition separates input ambiguity, representation loss, and readout mismatch. We construct recoverable views where perfect agreement and joint isotropic Gaussianity coexist with zero target information, and establish limits imposed by deterministic canonical anchors. Token log-loss provides a one-sided information-loss bound; a fixed-penalty ridge analysis shows why rank alone cannot determine prediction risk. These results motivate CANOPE, a nonautoregressive framework with ordered latent canvases, canonical-token supervision, and geometric regularization. On 40,000 validation sequences, latent-agreement (PL0) and token-grounded (PL2) have nearly identical pooled ranks but reach 13.5% and 98.8% positional Recall@1, respectively, under strong natural corruption when the correct target length is provided. On 3,930 LJSpeech validation utterances, frozen PL2 with a trained MatchaTTS readout yields 21.54% word error rate (WER) on corrupted text, versus 99.22% for frozen PL0, while end-to-end MatchaTTS reaches 10.93%. These results show that geometric regularity alone does not guarantee recoverable sequential content or effective downstream access in the text settings studied here.

发表机构

• Institut Teknologi Sepuluh Nopember（泗水理工学院）
• Avalon AI
• Universitas Indonesia（印度尼西亚大学）
• National University of Singapore（新加坡国立大学）

机构由 AI 辅助整理，请以论文原文为准。