Multimodal language models have demonstrated strong semantic understanding and reasoning over physiological and behavioral signals captured from diverse healthcare sensors. However, existing sensor-language models can only process a fixed number of sensor channels at a fixed sampling rate, making them difficult to adapt to new or different sensor configurations. This inflexibility prevents models from generalizing across heterogeneous health sensing ecosystems, where data spans high-frequency wearable streams to sparse, day-scale clinical signals. To bridge this gap, we introduce SLIP (Sensor Language-Informed Pretraining), an open-source framework for learning language-aligned representations that generalize across diverse sensor setups. SLIP integrates contrastive alignment with sensor-conditioned captioning, facilitating both discriminative understanding and generative reasoning. By repurposing a pretrained decoder-only language model via cross-attention and introducing a flexible patch embedder, SLIP transfers to new sensor configurations at inference without retraining, regardless of their native sampling rate or input length. Across 11 datasets, SLIP demonstrates superior performance in retrieval, signal captioning, and question answering. It achieves a 77.14% average linear-probing accuracy, a 3.56% improvement over the strongest baseline (SensorLM). Beyond classification, SLIP supports open-vocabulary sensor captioning and question answering without any architectural modifications. Our work offers broad implications for the development of sensor-language models, utilizing cross-modal supervision to ensure more robust generalizability in health sensing applications.
Muon Learns More Robust and Transferable Features than Adam
Yale University(耶鲁大学);National University of Singapore(新加坡国立大学);University of Chinese Academy of Sciences(中国科学院大学);Academy of Mathematics and Systems Science, CAS(中国科学院数学与系统科学研究院)
本文通过鲁棒性和可迁移性视角,证明 Muon 优化器相比 Adam 和 SGD 能学习到更鲁棒、更可迁移的特征,并通过理论分析支持了经验发现。
A Knowledge-Informed Large Language Model Framework for U.S. Nuclear Power Plant Shutdown Initiating Event Classification for Probabilistic Risk Assessment
University of Idaho(爱达荷大学);Idaho National Laboratory(爱达荷国家实验室)
Can Computational Reducibility Lead to Transferable Models for Graph Combinatorial Optimization?
Operations Research, Université de Montréal(运营研究,蒙特利尔大学);Mila – Quebec AI Institute(魁北克人工智能研究所);Valence Labs(Valence实验室);Dept. of Mathematics, Boise State University(数学系,博伊州立大学)
Biologically Informed Representation Learning for Robust Cross-Center Generalization of MALDI-TOF Mass Spectrometry
Universidad Carlos III de Madrid(卡洛斯三世大学);Instituto de Investigación Sanitaria Gregorio Marañón(格雷戈里奥·马拉尼翁卫生研究所);Hospital General Universitario Gregorio Marañón(格雷戈里奥·马拉尼翁大学总医院)
Label Informed Contrastive Pretraining for Node Importance Estimation on Knowledge Graphs
Tsinghua University(清华大学);Fuyao Institute of Technology(福耀科技大学);Chinese Academy of Sciences(中国科学院);Southern University of Science and Technology(南方科技大学);Fuzhou Institute of Data Technology(福州数据技术研究院)
Learning transferable human physiology from two million hours of sleep with SleepFM-2
Stanford University(斯坦福大学);BrainCapture;Technical University of Denmark(丹麦技术大学);Danish Center for Sleep Medicine(丹麦睡眠医学中心);Hvidovre Hospital(希维多夫医院);Pioneer Center for SMARTbiomed, National Research Center for the Working Environment(SMARTbiomed先锋中心,国家工作环境研究中心)
EyeMVP: OCT-Informed Fundus Representation Learning via Paired CFP--OCT Pretraining
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University(首都医科大学附属北京同仁医院北京同仁眼科中心);Liangxiang Hospital of Beijing Fangshan District, Capital Medical University(首都医科大学北京市房山区良乡医院);The Third People's Hospital of Dalian(大连市第三人民医院);National Clinical Research Center for Endocrine and Metabolic Diseases, The Second Xiangya Hospital of Central South University(中南大学湘雅二医院国家内分泌代谢病临床医学研究中心);The Central Hospital of Baoji City(宝鸡市中心医院);Wuxi No.2 People's Hospital, Affiliated Wuxi Clinical College of Nantong University(南通大学附属无锡临床学院无锡市第二人民医院);Shenzhen Eye Hospital, Southern Medical University(南方医科大学深圳眼科医院);Beijing Friendship Hospital, Capital Medical University(首都医科大学附属北京友谊医院)
Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation
University of Texas at Austin(德克萨斯大学奥斯汀分校);Zhejiang University(浙江大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Tencent AI Lab Seattle(腾讯AI实验室西雅图)
Representation Potentials of Foundation Models for Multimodal Alignment: A Survey
Department of Electrical and Computer Engineering, Northeastern University(东北大学电气与计算机工程系);Khoury College of Computer Science, Northeastern University(东北大学科赫里计算机科学学院)