尽管变压器在性能上表现出色，但其内部决策过程仍然缺乏透明性。我们研究了是否可以通过架构约束来通过架构流独立性强制实现可解释性：通过维护一个承载符号结构的标记流和在分离的流中保持上下文语义，在处理过程中始终保持独立可观察性，整合延迟到输出。我们通过晚期融合架构（LFA）验证了这一原则，该架构通过所有最终层实现了可解释的符号头，而标准变压器在第六层的三分之一处出现溶解；我们通过引入令牌位置依赖性分数（PDS）量化这一影响，PDS_max分别为0.276和0.058。关键的是，干预实验表明功能模块化：抑制LFA的近期头会导致最小的语义损害（Cohen's d = -0.158）与基线中的灾难性纠缠（d = -0.672）相比。LFA表明，架构约束提高了底层学习机制，平均稳定性为42%，而基线比较为19%和11%，极端情况从LFA的最佳配对（6个12个头位置不变）的50%下降到0%的完全崩溃。通过防止早期纠缠，架构独立性使模型朝着语义理解而非位置启发式方向发展，确立了可解释性作为通过结构约束可强制实施的架构设计标准，而不是事后分析。
英文摘要：
While transformers achieve strong performance, their internal decision-making processes remain opaque. We investigate whether architectural constraints can enforce interpretability by design through architectural stream independence: maintaining a token stream (carrying symbolic structure) and contextual semantics in separated streams that remain independently observable throughout processing, with integration delayed until output. We validate this principle through the Late Fusion Architecture (LFA), which demonstrates interpretable symbolic heads through all the final layers, while standard transformers show dissolution by the third of six layers; we quantify this effect by introducing the Token-Position Dependence Score (PDS), with $PDS_{max}$ = 0.276 and 0.058, respectively. Crucially, intervention experiments demonstrate functional modularity: suppressing LFA's recency heads causes minimal semantic damage (Cohen's d = -0.158) versus catastrophic entanglement in baselines (d = -0.672). LFA demonstrates that architectural constraints improve underlying learning mechanisms, averaging 42% stability versus 19% and 11% for baseline comparisons, with extremes from 50% on LFA's best pairs (6 of 12 heads position-invariant) down to 0% complete collapse in over-constrained cases. By preventing premature entanglement, architectural independence steers models toward semantic understanding over positional heuristics, establishing interpretability as an architectural design criterion enforceable through structural constraints rather than post-hoc analysis.
Time-Aware and Transition-Semantic Graph Neural Networks for Interpretable Predictive Business Process Monitoring
College of Computing and Mathematical Sciences, Khalifa University（卡利法大学计算与数学科学学院）；Department of Computer Science, Università degli Studi di Milano（米兰大学计算机科学系）
Multi-objective Optimization in CPU Design Space Exploration: Attention is All You Need
State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所处理器重点实验室）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；University of Electronic Science and Technology of China（电子科技大学）；School of Integrated Circuits, Peking University（北京大学集成电路学院）
