Large language models (LLMs) are advanced AI systems applied across various domains, including NLP, information retrieval, and recommendation systems. Despite their adaptability and efficiency, LLMs have not been extensively explored for signal processing tasks, particularly in the domain of global navigation satellite system (GNSS) interference monitoring. GNSS interference monitoring is essential to ensure the reliability of vehicle localization on roads, a critical requirement for numerous applications. However, GNSS-based positioning is vulnerable to interference from jamming devices, which can compromise its accuracy. The primary objective is to identify, classify, and mitigate these interferences. Interpreting GNSS snapshots and the associated interferences presents significant challenges due to the inherent complexity, including multipath effects, diverse interference types, varying sensor characteristics, and satellite constellations. In this paper, we extract features from a large GNSS dataset and employ LLaVA to retrieve relevant information from an extensive knowledge base. We employ prompt engineering to interpret the interferences and environmental factors, and utilize t-SNE to analyze the feature embeddings. Our findings demonstrate that the proposed method is capable of visual and logical reasoning within the GNSS context. Furthermore, our pipeline outperforms state-of-the-art machine learning models in interference classification tasks.
LLM-Guided Indoor Navigation with Multimodal Map Understanding
Institute of Information Science and Technologies (ISTI)(信息科学与技术研究所);National Research Council of Italy (CNR)(意大利国家研究理事会);Department of Computer Science(计算机科学系);University of Pisa(比萨大学);University College Dublin (UCD)(都柏林大学学院);School of Computer Science(计算机科学学院)
Active Sensing with Meta-Reinforcement Learning for Emitter Localization from RF Observations
Fraunhofer Institute for Integrated Circuits IIS(弗劳恩霍夫集成电路研究所);Positioning Systems Lab, University of Technology Nürnberg (UTN)(定位系统实验室,图恩大学);Center for Artificial Intelligence, Technical University of Applied Sciences Würzburg-Schweinfurt(人工智能中心,应用技术大学魏玛-施魏尔堡)
A Step Toward Federated Pretraining of Multimodal Large Language Models
MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统实验室);King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);Pengcheng Laboratory(鹏城实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)(中国科学院大学人工智能学院)
MINT: Multimodal Instruction Tuning with Multimodal Interaction Grouping
University of California, San Diego(加州大学圣地亚哥分校);Johns Hopkins University(约翰霍普金斯大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Massachusetts Institute of Technology(麻省理工学院)
本文提出 MINT 任务分组策略,按多模态交互类型组织指令微调任务,以学习可迁移技能并减少任务干扰,在泛化与专业化间取得更优平衡。
MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts for Multimodal Knowledge Graph Completion
Beijing University of Posts and Telecommunications(北京邮电大学);Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学);Communication University of China(中国传媒大学)
Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models
Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区);Zhongguancun Academy(中关村学院);University of Chinese Academy of Sciences(中国科学院大学);Nanyang Technological University(南洋理工大学);Department of Mechanical Engineering, Imperial College London(伦敦帝国理工学院机械工程系)
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳)计算机科学与技术学院);School of Computer Science, University of Sydney(悉尼大学计算机学院);Alibaba Group(阿里巴巴集团)
Model Composition for Multimodal Large Language Models
Tsinghua University(清华大学);Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院);Institute of Intelligent Computing, Alibaba Group(阿里巴巴集团智能计算研究院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
JSR-GFNet: Jamming-to-Signal Ratio-Aware Dynamic Gating for Interference Classification in future Cognitive Global Navigation Satellite Systems
National Key Laboratory of Wireless Communications, University of Electronic Science and Technology of China(中国电子科技大学无线通信国家重点实验室);University of Electronic Science and Technology of China(中国电子科技大学);Shanghai Aerospace Electronic Technology Institute(上海航天电子技术研究所);Shanghai Key Laboratory of Collaborative Computing in Spacial Heterogeneous Networks (CCSN)(上海空间异构网络协同计算重点实验室);Anhui Science and Technology University(安徽科技大学);University of Oxford(牛津大学);Shanghai Xiaoyuan Innovation center(上海小元创新中心)
Improving Multimodal Learning Balance and Sufficiency through Data Remixing
School of Computer Science and Engineering, Southeast University, Nanjing, China(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及交叉应用重点实验室);College of Computer Science, Beijing University of Technology, Beijing, China(北京理工大学计算机学院)
Liaohe-CobotMagic-PnP: an Imitation Learning Dataset of Intelligent Robot for Industrial Applications
School of Information Science and Engineering, Shandong Normal University(信息科学与工程学院,山东师范大学);School of Advanced Technology, Xi’an Jiaotong-Liverpool University(先进技术学院,西安交通大学利物浦大学);Liaohe Laboratory, Innovation Institute of Intelligent Robotics Shenyang Co., Ltd.(辽河实验室,沈阳智能机器人创新研究院有限公司);School of Information Engineering, Shenyang University of Chemical Technology(信息工程学院,沈阳化工大学);Shenyang Institute of Automation, Chinese Academy of Sciences(沈阳自动化研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学)