arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.11950cs.AIcs.LG
收藏

通过语言引导预训练学习可迁移的传感器模型

Learning Transferable Sensor Models via Language-Informed Pretraining

Yuliang Chen, Arvind Pillai, Yu Yvonne Wu, Sudarshan Regmi, Tess Z. Griffin, Lisa Marsch, Michael V. Heinz, Nicholas C. Jacobson, Andrew Campbell

AI总结:

SLIP通过语言引导预训练学习可迁移的传感器模型,整合对比对齐与传感器条件标注,支持不同时间分辨率和输入长度,实现跨领域任务的高效性能。

摘要:

现代传感系统生成大量未标记的多变量时间序列数据。这种大量未标记的数据使自监督学习(SSL)成为学习可迁移表示的自然方法。然而,大多数现有方法优化于重建或预测目标,往往无法捕捉下游分类和推理任务所需的语义结构。尽管最近的传感器-语言对齐方法通过标注和零样本转移提高语义泛化能力,但它们局限于固定的传感器配置,如预定义的通道集、信号长度或时间分辨率,这阻碍了跨领域应用。为解决这些不足,我们引入SLIP(传感器-语言引导预训练),一个开源框架,用于学习在不同传感器设置下通用的语义表示。SLIP整合对比对齐与传感器条件标注,促进判别理解和生成推理。通过交叉注意力重用预训练的解码器-only语言模型,并引入优雅且灵活的补丁嵌入器,SLIP在推理时支持不同的时间分辨率和可变长度输入,无需额外微调。在11个数据集上,SLIP在零样本迁移、信号标注和问答任务中表现出色。它实现了77.14%的平均线性探测准确率,比强基线高出5.93%,并在基于传感器的问答任务中达到64.83%的准确率。

英文摘要:

Multimodal language models have demonstrated strong semantic understanding and reasoning over physiological and behavioral signals captured from diverse healthcare sensors. However, existing sensor-language models can only process a fixed number of sensor channels at a fixed sampling rate, making them difficult to adapt to new or different sensor configurations. This inflexibility prevents models from generalizing across heterogeneous health sensing ecosystems, where data spans high-frequency wearable streams to sparse, day-scale clinical signals. To bridge this gap, we introduce SLIP (Sensor Language-Informed Pretraining), an open-source framework for learning language-aligned representations that generalize across diverse sensor setups. SLIP integrates contrastive alignment with sensor-conditioned captioning, facilitating both discriminative understanding and generative reasoning. By repurposing a pretrained decoder-only language model via cross-attention and introducing a flexible patch embedder, SLIP transfers to new sensor configurations at inference without retraining, regardless of their native sampling rate or input length. Across 11 datasets, SLIP demonstrates superior performance in retrieval, signal captioning, and question answering. It achieves a 77.14% average linear-probing accuracy, a 3.56% improvement over the strongest baseline (SensorLM). Beyond classification, SLIP supports open-vocabulary sensor captioning and question answering without any architectural modifications. Our work offers broad implications for the development of sensor-language models, utilizing cross-modal supervision to ensure more robust generalizability in health sensing applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑