Powerformer: A Section-adaptive Transformer for Power Flow Adjustment
State Key Laboratory of Blockchain & Data Security, Zhejiang University（浙江大学区块链与数据安全国家重点实验室）；College of Electrical Engineering, Zhejiang University（浙江大学电气工程学院）；State Grid Shandong Electric Power Company（国网山东省电力公司）
Kaixuan Chen, Wei Luo, Shunyu Liu, Yaoquan Wei, Yihe Zhou, Yunpeng Qing, Quan Zhang, Jie Song, Mingli Song
In this paper, we present a novel transformer architecture tailored for learning robust power system state representations, which strives to optimize power dispatch for the power flow adjustment across different transmission sections. Specifically, our proposed approach, named Powerformer, develops a dedicated section-adaptive attention mechanism, separating itself from the self-attention used in conventional transformers. This mechanism effectively integrates power system states with transmission section information, which facilitates the development of robust state representations. Furthermore, by considering the graph topology of power system and the electrical attributes of bus nodes, we introduce two customized strategies to further enhance the expressiveness: graph neural network propagation and multi-factor attention mechanism. Extensive evaluations are conducted on three power system scenarios, including the IEEE 118-bus system, a realistic 300-bus system in China, and a large-scale European system with 9241 buses, where Powerformer demonstrates its superior performance over several baseline methods.
Context-Aware Rule Mining Using a Dynamic Transformer-Based Framework
University of Minnesota（明尼苏达大学）；Cornell University（康奈尔大学）；Northeastern University（东北大学）；University of Rochester（罗切斯特大学）；University of Miami（迈阿密大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）
SpecFormer: Guarding Vision Transformer Robustness via Maximum Singular Value Penalization
City University of Hong Kong（香港城市大学）；Microsoft Research Asia（微软亚洲研究院）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；William & Mary（威廉玛丽学院）
WISCA: A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling
Meituan, Beijing, China（美团，北京，中国）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学，北京，中国）；Hong Kong University of Science and Technology, Hong Kong SAR, China（香港科技大学，香港特别行政区，中国）；Xiamen University, Xiamen, China（厦门大学，厦门，中国）
Fusion: A Framework for Unified Sequential Token AdaptatIon in VisiOn TraNsformers
Brandenburg University of Technology Cottbus-Senftenberg（科特布斯-森夫滕贝格勃兰登堡工业大学）；University of Zanjan（赞詹大学）；Humboldt University（洪堡大学）；Tallinn University of Technology（塔林理工大学）
Imputation-free transformer learning enables robust Alzheimer's disease prediction and calibrated uncertainty quantification across heterogeneous clinical cohorts
Platform of Bioinformatics（生物信息平台）；Lausanne University Hospital（洛桑大学医院）；Faculty of Biology and Medicine（生物学与医学学院）；University of Lausanne（洛桑大学）；Leenaards Memory Centre（Leenaards记忆中心）；Department of Clinical Neurosciences（临床神经科学系）
Native Hybrid Attention for Efficient Sequence Modeling
Tsinghua University（清华大学）；Shanghai AI Laboratory（上海人工智能实验室）；The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；The Chinese University of Hong Kong（香港中文大学）
