arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

TriCLE:面向边缘部署的细粒度聚类的三模态视觉-语言推理

TriCLE: Tri-Modal Vision-Language Reasoning for Edge-Deployed Fine-Grained Clustering

Kishor Datta Gupta, Md. Mahfuzur Rahman, Fahad Rahman, Ahmed Rafi Hasan, Faysal Mehrab Chowdhury, Mohd Ariful Haque, Roy George

arXiv 2608.04175首次发表:更新:

发表机构

Clark Atlanta University; United International University; North South University(克拉克亚特兰大大学; 国际大学; 北南大学)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

TriCLE是面向边缘部署的三模态视觉-语言系统,通过生成伪热视图和伪激光雷达深度,结合对齐策略实现细粒度飞机聚类,适配8GB内存边缘设备,验证与测试均取得良好性能。

AI 中文摘要

用于航空观测的边缘平台必须在内存有限、计算资源有限且连接间歇性的条件下解读飞机图像。这种设置对于仅使用RGB的标准识别模型和通用视觉-语言模型来说颇具挑战,尤其是在缺少校准后的热红外(FLIR)和激光雷达(LiDAR)飞机数据时。我们提出TriCLE,这是一种面向应用的三模态视觉-语言系统,用于在边缘约束下进行飞机分类分组。TriCLE从单张RGB飞机图像生成结构保留的FLIR风格热视图和伪激光雷达深度投影,随后将对齐后的视图与任务指令在紧凑的Qwen3-VL主干中进行融合。该模型基于推进系统、机身系列、尺寸、设计时代和构型的专家飞机分类法进行对齐,因此其输出反映的是与工程相关的相似性,而非仅表面外观。我们评估了监督微调、旋转保留的SFT以及三种策略对齐策略:GRPO、GSPO和DAPO。序列级GSPO展现出最强的验证性能,达到88.33%的验证准确率和0.91的加权F1值。在保留的飞机测试分区上,GSPO实现了78.00%的准确率和0.793的加权F1值,同时保持了94.00%的可解析输出格式。经过4位量化和注意力-内存优化后,对齐后的4B模型适配8GB部署目标,处理每个三模态三元组耗时1.48秒。这些结果表明TriCLE是一款实用的可解释、边缘可行的飞机分组原型,同时强调需要在真实对齐的热红外和激光雷达传感器流上进行进一步验证。

英文摘要

Edge platforms used for aerial observation must interpret aircraft imagery under limited memory, limited compute, and intermittent connectivity. This setting is difficult for standard RGB-only recognition models and general-purpose vision-language models, especially when calibrated thermal and LiDAR aircraft data are unavailable. We present TriCLE, an application-oriented tri-modal vision-language system for aircraft taxonomic grouping under edge constraints. From a single RGB aircraft image, TriCLE generates a structure-preserving FLIR-style thermal view and a pseudo-LiDAR depth projection, then fuses the aligned views with task instructions in a compact Qwen3-VL backbone. The model is aligned to an expert aircraft taxonomy based on propulsion, airframe family, size, design era, and configuration, so its outputs reflect engineering-relevant similarity rather than only surface appearance. We evaluate supervised fine-tuning, rotation-preserving SFT, and three policy-alignment strategies: GRPO, GSPO, and DAPO. Sequence-level GSPO gives the strongest validation performance, reaching 88.33\% validation accuracy and 0.91 weighted F1 on valid aircraft outputs. On a held-out aircraft test partition, GSPO achieves 78.00\% accuracy and 0.793 weighted F1 while preserving 94.00\% parseable output formatting. After 4-bit quantization and attention-memory optimization, the aligned 4B model fits an 8GB deployment target and processes each tri-modal triplet in 1.48 seconds. These results support TriCLE as a practical prototype for interpretable, edge-feasible aircraft grouping, while emphasizing the need for further validation on real aligned thermal and LiDAR sensor streams.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑