EventCLIP：将CLIP适配用于基于事件的目标识别
EventCLIP: Adapting CLIP for Event-based Object Recognition
AI总结：
EventCLIP通过将CLIP适配到事件数据，利用二维网格表示和特征适配器，在多个数据集上实现最先进的少样本和全量微调事件目标识别性能。
摘要：
最近在零样本和少样本分类方面的进展很大程度上依赖于预训练的视觉-语言模型（如CLIP）的成功。由于缺乏大规模数据集，为事件相机数据训练此类模型仍然不可行。因此，将现有的视觉-语言模型跨模态适配到事件视觉是一个重要的研究挑战。在这项工作中，我们引入了EventCLIP，一种利用CLIP进行零样本和少样本基于事件的目标识别的新方法。我们首先通过将原始事件转换为基于二维网格的表示，将CLIP的图像编码器推广到事件数据。为了进一步提升性能，我们提出了一种特征适配器，用于聚合事件帧上的时间信息，并细化文本嵌入以更好地与视觉输入对齐。我们在N-Caltech、N-Cars和N-ImageNet数据集上评估了EventCLIP，取得了最先进的少样本性能。当在整个数据集上微调时，我们的方法优于所有现有的事件分类器。此外，我们探索了EventCLIP的实际应用，包括鲁棒事件分类和无标签事件识别，在这些任务中，我们的方法超越了先前专门为这些任务设计的基线。
英文摘要：
Recent advances in zero-shot and few-shot classification heavily rely on the success of pre-trained vision-language models (VLMs) such as CLIP. Due to a shortage of large-scale datasets, training such models for event camera data remains infeasible. Thus, adapting existing VLMs across modalities to event vision is an important research challenge. In this work, we introduce EventCLIP, a novel approach that utilizes CLIP for zero-shot and few-shot event-based object recognition. We first generalize CLIP's image encoder to event data by converting raw events to 2D grid-based representations. To further enhance performance, we propose a feature adapter to aggregate temporal information over event frames and refine text embeddings to better align with the visual inputs. We evaluate EventCLIP on N-Caltech, N-Cars, and N-ImageNet datasets, achieving state-of-the-art few-shot performance. When fine-tuned on the entire dataset, our method outperforms all existing event classifiers. Moreover, we explore practical applications of EventCLIP including robust event classification and label-free event recognition, where our approach surpasses previous baselines designed specifically for these tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
