Kan Extension Transformers: A Categorical Unification of Attention, Diffusion, and Predict-Detach Self-Conditioning
Kan扩展变换器:注意力、扩散和预测-分离自条件的范畴统一
机构 * Adobe Research(Adobe研究院) ; University of Massachusetts(马萨诸塞大学) ; Amherst(阿默斯特)
专题命中 扩散模型 :diffusion(title)
AI总结 提出Kan扩展变换器(KETs)作为多种Transformer实现的统一范畴框架,将Transformer层视为加权结构化扩展算子,并通过预测-分离机制实现有效的自条件化,实验表明预测-分离机制比改变邻域族带来更大性能提升。
Comments Clarified that predict-detach is strictly autoregressive only under a target-relative prefix-measurability condition; restricted exact Kan-extension claims to representable enriched cases; and clarified experimental coverage across depths 2, 8, 16 and widths 64, 256. A detailed change log appears in the supplement