VSLLaVA: a pipeline of large multimodal foundation model for industrial vibration signal analysis
State Key Laboratory of Tribology, Department of Mechanical Engineering, Tsinghua University（摩擦学国家重点实验室，清华大学机械工程系）；Department of Statistics and Data Science, Yale University（耶鲁大学统计与数据科学系）
While Large Multimodal Models (LMMs) excel in general multimodal tasks, they lack the domain-specific knowledge for industrial vibration signal analysis. This paper introduces VSLLaVA, a comprehensive pipeline that utilizes expert knowledge-guided instruction tuning and evaluation to create an end-to-end LMM for signal analysis. To achieve this, we construct a novel Signal-Question-Answer (SQA) dataset using an expert rule-based signal generator. This dataset facilitates a two-stage learning procedure. The first step is efficient instruction fine-tuning with Low-Rank Adaptation (LoRA), which imparts specialized signal identification capabilities. Subsequently, we designed a tailored Group Relative Policy Optimization (GRPO) to refine the reasoning capabilities and enhance classification robustness. Then, a dual-mode evaluation framework is proposed, combining an LLM referee with expert rules for semantic assessment using quantitative metrics for numerical and textual accuracy, which reveals that VSLLaVA significantly improves performance in signal type identification and parameter analysis, and makes progress in the identification and parameter analysis of fault-related signals. This research demonstrates a viable approach for developing specialized foundational models for complex industrial applications and marks a transition from conventional task-specific systems to a cohesive, interactive foundational model.
BearLLM: A Prior Knowledge-Enhanced Bearing Health Management Framework with Unified Vibration Signal Representation
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；Liaoning Liaohe Laboratory（辽宁辽河实验室）；Key Laboratory on Intelligent Detection and Equipment Technology of Liaoning Province（辽宁省智能检测与装备技术重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
Identification of fault frequency variation in the envelope spectrum in the vibration-based local damage detection in possible changing load/speed conditions
