arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.13634astro-ph.IM
收藏

结合注意力机制与Vision Transformers的增强天文源分类

Enhanced Astronomical Source Classification with Integration of Attention Mechanisms and Vision Transformers

Srinadh Reddy Bhavanam, Sumohana S. Channappayya, P. K. Srijith, Shantanu Desai

AI总结:

本文提出在MargNet天文分类器中引入注意力机制和Vision Transformer处理图像数据,结果表明注意力增强型CNN略优于传统模型,而基于ViT的混合模型在保持相当准确率的同时更为轻量易训。

摘要:

天体的准确分类对于增进我们对宇宙的理解至关重要。MargNet是一种最近开发的基于深度学习的分类器,应用于SDSS DR16数据集,利用测光数据将恒星、类星体和致密星系区分开来。MargNet采用堆叠架构,结合用于图像建模的卷积神经网络(CNN)和用于建模测光参数的人工神经网络(ANN)。在本研究中,我们提出通过引入注意力机制和基于Vision Transformer(ViT)的模型来处理图像数据,从而提升MargNet的性能。注意力机制使模型能够聚焦于相关特征并捕捉图像内的复杂模式,有效区分不同类别的天体。此外,我们利用了ViT,这是一种基于Transformer的深度学习架构,因在图像分类任务中表现卓越而闻名。我们利用ViT捕捉全局依赖关系和上下文信息的能力,增强了模型对复杂天文图像的理解。我们的方法使用了一个精选数据集,包含240,000个致密天体和150,000个暗弱天体。模型直接从数据中学习分类,最大限度地减少了人工干预。此外,我们探索了将ViT作为一种混合架构,将测光特征和图像一起作为输入来预测天体。我们的结果表明,所提出的引入注意力机制增强的MargNet CNN在性能上略优于传统的MargNet和所提出的基于ViT的MargNet模型。此外,基于ViT的混合模型成为最轻量且易于训练的模型,其分类准确率与表现最佳的注意力增强型MargNet相似。

英文摘要:

Accurate classification of celestial objects is essential for advancing our understanding of the universe. MargNet is a recently developed deep learning-based classifier applied to SDSS DR16 dataset to segregate stars, quasars, and compact galaxies using photometric data. MargNet utilizes a stacked architecture, combining a Convolutional Neural Network (CNN) for image modelling and an Artificial Neural Network (ANN) for modelling photometric parameters. In this study, we propose enhancing MargNet's performance by incorporating attention mechanisms and Vision Transformer (ViT)-based models for processing image data. The attention mechanism allows the model to focus on relevant features and capture intricate patterns within images, effectively distinguishing between different classes of celestial objects. Additionally, we leverage ViTs, a transformer-based deep learning architecture renowned for exceptional performance in image classification tasks. We enhance the model's understanding of complex astronomical images by utilizing ViT's ability to capture global dependencies and contextual information. Our approach uses a curated dataset comprising 240,000 compact and 150,000 faint objects. The models learn classification directly from the data, minimizing human intervention. Furthermore, we explore ViT as a hybrid architecture that uses photometric features and images together as input to predict astronomical objects. Our results demonstrate that the proposed attention mechanism augmented CNN in MargNet marginally outperforms the traditional MargNet and the proposed ViT-based MargNet models. Additionally, the ViT-based hybrid model emerges as the most lightweight and easy-to-train model with classification accuracy similar to that of the best-performing attention-enhanced MargNet.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑