ChaosProbe：面向冻结Transformer输入嵌入空间的神经混沌视角
ChaosProbe: A Neurochaotic Lens on Frozen Transformer Input-Embedding Spaces
浏览论文内容
中文总结 AI 辅助
本研究提出ChaosProbe方法，通过神经混沌变换生成冻结Transformer输入嵌入空间的响应指纹，实验验证其可有效揭示不同预训练Transformer嵌入空间的结构关联。
中文摘要 AI 辅助
人们通常通过Transformer模型的基准性能、生成质量或下游任务表现来理解它们，但也可在上下文计算或特定任务适配前，通过受控确定性探针的响应来研究其冻结的输入嵌入空间。基于这种响应视角，我们提出ChaosProbe，一种受神经混沌启发的确定性方法，用于构建冻结Transformer输入嵌入空间的响应指纹。对于每个提示级嵌入矩阵，ChaosProbe应用基于混沌轨迹的变换，并结合互补的表示级度量来总结其激发率和熵通道响应，为每个模型生成长度固定的特征。在包含80个中性提示和四个预训练模型（GPT-2、DistilGPT2、BERT-base-uncased、RoBERTa-base）的有限概念验证研究中，皮尔逊相关系数、斯皮尔曼相关系数和余弦相似度均能恢复所有四个同家族近邻分配及两个预期的家族间配对；欧氏距离恢复四个分配中的三个及两个家族间配对中的一个。配对自助重采样支持皮尔逊和斯皮尔曼配对在观测提示集上的稳定性，特征有效性检查显示恒定或崩溃的响应未主导所报告的指纹。这些结果提供了依赖于群体的概念验证，表明确定性神经混沌响应特征可揭示冻结Transformer输入嵌入空间的广泛结构。
英文摘要
Transformer models are most often understood through what they do: their benchmark performance, generation quality, or behavior on downstream tasks. Yet frozen transformer input-embedding spaces may also be examined through their responses to a controlled deterministic probe before contextual computation or task-specific adaptation. Guided by this response-based view, we introduce ChaosProbe, a deterministic neurochaos-inspired method for constructing response-based fingerprints of frozen transformer input-embedding spaces. For each prompt-level embedding matrix, ChaosProbe applies a chaotic trajectory-based transformation and summarizes its Firing Rate and Entropy channel responses with complementary representation-level measures, producing a fixed-length signature for each model. In a bounded proof-of-concept study of $80$ neutral prompts and four pretrained models---GPT-2, DistilGPT2, BERT-base-uncased, and RoBERTa-base---Pearson correlation, Spearman correlation, and cosine similarity each recover all four same-family nearest-neighbor assignments and both expected mutual family pairs. Euclidean distance recovers three of the four assignments and one of the two mutual family pairs. Paired bootstrap resampling supports the stability of the Pearson and Spearman pairings over the observed prompt set, and signature-validity checks show that constant or collapsed responses do not dominate the reported fingerprints. These results provide a cohort-dependent proof of concept that deterministic neurochaotic response signatures can expose broad structure among frozen transformer input-embedding spaces.
发表机构
- Amrita Vishwa Vidyapeetham（阿米塔大学）
- National Institute of Advanced Studies (NIAS)（国家高级研究院）
机构由 AI 辅助整理，请以论文原文为准。
