大语言模型能否捕捉事件信号？探索纯零样本基于事件的识别
Can Large Language Models Grasp Event Signals? Exploring Pure Zero-Shot Event-based Recognition
AI总结：
本研究首次探索大语言模型对基于事件视觉内容的理解能力，证明其无需额外训练或结合CLIP微调即可实现纯零样本基于事件的识别，实验显示经精心提示增强的LLMs性能显著提升，GPT-4o在N-ImageNet上准确率超现有最优方法五个数量级。
摘要：
基于事件的零样本目标识别近期取得了可喜进展，但这类方法严重依赖大量训练，且固有地受限于CLIP的特性。据我们所知，本研究首次探索了大语言模型（LLMs）对基于事件的视觉内容的理解能力。我们证明，LLMs无需额外训练或与CLIP联合微调，即可实现基于事件的目标识别，有效达成纯零样本基于事件的识别。我们特别评估了GPT-4o/4turbo及另外两款开源LLM直接识别基于事件的视觉内容的能力，在三个基准数据集上开展了大量实验，系统评估了这些模型的识别准确率。结果表明，LLMs尤其是在精心设计的提示增强下，能显著提升基于事件的零样本识别性能。值得注意的是，GPT-4o表现优于所有对比模型，且在N-ImageNet上的识别准确率比当前最优的基于事件的零样本方法高出五个数量级。本文的实现代码可在https://github.com/ChrisYu-Zz/Pure-event-based-recognition-based-LLM获取。
英文摘要：
Recent advancements in event-based zero-shot object recognition have demonstrated promising results. However, these methods heavily depend on extensive training and are inherently constrained by the characteristics of CLIP. To the best of our knowledge, this research is the first study to explore the understanding capabilities of large language models (LLMs) for event-based visual content. We demonstrate that LLMs can achieve event-based object recognition without additional training or fine-tuning in conjunction with CLIP, effectively enabling pure zero-shot event-based recognition. Particularly, we evaluate the ability of GPT-4o / 4turbo and two other open-source LLMs to directly recognize event-based visual content. Extensive experiments are conducted across three benchmark datasets, systematically assessing the recognition accuracy of these models. The results show that LLMs, especially when enhanced with well-designed prompts, significantly improve event-based zero-shot recognition performance. Notably, GPT-4o outperforms the compared models and exceeds the recognition accuracy of state-of-the-art event-based zero-shot methods on N-ImageNet by five orders of magnitude. The implementation of this paper is available at \url{https://github.com/ChrisYu-Zz/Pure-event-based-recognition-based-LLM}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
