arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.07469astro-ph.IM
收藏

迈向射电天文源分析的视觉-语言助手

Toward Vision-Language Assistants for Radio Astronomical Source Analysis

INAF - Osservatorio Astrofisico di Catania(意大利国家天体物理研究所 - 卡塔尼亚天文台)

S. Riggi

AI总结:

本文提出radio-llava,一种针对射电天文的视觉-语言助手,通过指令微调提升多任务性能,但发现专用视觉模型仍更优,且微调导致通用任务性能下降。

摘要:

视觉-语言模型(VLMs)最近在通用任务推理中展现出潜力,但其在领域特定科学工作流中的适用性仍鲜有探索。本文评估了一系列开源和商用VLMs在六个与射电天文相关的任务上的表现,如源形态分类。我们还引入了radio-llava,一种基于LLaVA架构的微调多模态助手,并通过指令微调适应射电领域。在零样本模式下,商用模型如GPT-4.1在大多数射电基准测试中优于开源VLMs。然而,radio-llava在几乎所有任务上均显著优于基础LLaVA和商用模型。尽管有这些进步,专门的视觉-only模型在整体上仍表现更优。此外,我们发现微调会引入对通用多模态任务的灾难性遗忘,性能下降可达40%,这可以通过更多样化的训练数据集或浅层微调部分缓解。

英文摘要:

Vision-language models (VLMs) have recently shown promise in general-purpose reasoning tasks, yet their applicability to domain-specific scientific workflows remains largely unexplored. In this work, we evaluated a series of open-weight and commercial VLMs on six tasks relevant to radio astronomy, such as source morphology classification. We also introduced radio-llava, a fine-tuned multimodal assistant built on the LLaVA architecture and adapted for the radio domain through instruction fine-tuning. In zero-shot mode, commercial models like GPT-4.1 outperform open-weight VLMs on most radio benchmarks. However, radio-llava significantly improves upon both base LLaVA and commercial models across nearly all tasks. Despite these gains, specialized vision-only models still deliver substantially better performance across the board. Additionally, we observed that fine-tuning introduces catastrophic forgetting on general multimodal tasks, with performance drops up to 40% that can be partly mitigated with a more diverse training dataset or shallow fine-tuning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑