单层Transformer可证明学习多类环境下的单最近邻算法
One-Layer Transformer Provably Learns Multiclass One-Nearest Neighbor in Context
浏览论文内容
中文总结 AI 辅助
该研究将单层Transformer与最近邻分类器的等价性从二分类扩展至多分类，证明带argmax头的单层Transformer等价于多分类单最近邻，填补了前期研究的空白。
中文摘要 AI 辅助
我们将近期关于单层Transformer与最近邻分类器在二分类场景下等价性的研究扩展至多分类情形。通过利用单纯形编码，我们证明带有argmax分类头的单层Transformer在多分类场景下表现与单最近邻分类器完全一致。这填补了前期研究留下的空白，前期研究的多分类结果依赖非标准的基于舍入的方法，而非实践中常用的argmax分类头。
英文摘要
We extend recent work establishing an equivalence between one-layer transformers and nearest-neighbor classifiers in the binary setting to the multiclass case. By leveraging the simplex encoding, we show that one-layer transformers with an argmax classification head behave identically to a one-nearest-neighbor classifier in the multiclass setting. This closes a gap left by prior work, whose multiclass result relied on a non-standard rounding-based approach rather than the typical argmax head used in practice.
发表机构
- James B. Conant High School（詹姆斯·B·柯南特高中）
- Illinois Institute of Technology（伊利诺伊理工大学）
机构由 AI 辅助整理，请以论文原文为准。
