The Automated Audio Captioning (AAC) task asks models to generate natural language descriptions of an audio input. Evaluating these machine-generated audio captions is a complex task that requires considering diverse factors, among them, auditory scene understanding, sound-object inference, temporal coherence, and the environmental context of the scene. While current methods focus on specific aspects, they often fail to provide an overall score that aligns well with human judgment. In this work, we propose CLAIR-A, a simple and flexible method that leverages the zero-shot capabilities of large language models (LLMs) to evaluate candidate audio captions by directly asking LLMs for a semantic distance score. In our evaluations, CLAIR-A better predicts human judgements of quality compared to traditional metrics, with a 5.8% relative accuracy improvement compared to the domain-specific FENSE metric and up to 11% over the best general-purpose measure on the Clotho-Eval dataset. Moreover, CLAIR-A offers more transparency by allowing the language model to explain the reasoning behind its scores, with these explanations rated up to 30% better by human evaluators than those provided by baseline methods. CLAIR-A is made publicly available at https://github.com/DavidMChan/clair-a.
Agri-CPJ: A Training-Free Explainable Framework for Agricultural Pest Diagnosis Using Caption-Prompt-Judge and LLM-as-a-Judge
Business School, Shandong University of Technology（山东理工大学商学院）；Faculty of Data Science, City University of Macau（澳门城市大学数据科学学院）；Guangdong Institute of Intelligent Science and Technology（广东智能科学与技术研究院）；Macau Millennium College（澳门 millennium 学院）；Department of Computer and Information Science, University of Macau（澳门大学计算机与信息科学系）
CPJ: Explainable Agricultural Pest Diagnosis via Caption-Prompt-Judge with LLM-Judged Refinement
Business School, Shandong University of Technology, Shandong, China（山东科技大学商学院）；Institute of International Language Services Studies, Macau Millennium College, Macau SAR, China（国际语言服务研究所，澳门 Millennium 学院）
TextTIGER: Text-based Intelligent Generation with Entity Prompt Refinement for Text-to-Image Generation
Nara Institute of Science and Technology（奈良科学技術研究所）；The University of Tokyo（东京大学）；Chungnam National University（Chungnam国立大学）；Institute of Science Tokyo（东京科学研究所）
CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era
National Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；The University of Hong Kong（香港大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
CLIPScore: A Reference-free Evaluation Metric for Image Captioning
Allen Institute for AI（艾伦人工智能研究所）；University of Washington（华盛顿大学）
图像描述传统上依赖于基于参考的自动评估，其中将机器字幕与人类书写的字幕进行比较。这与人类评估字幕质量的无参考方式形成对比。在本文中，我们报告了令人惊讶的实证发现，CLIP（Radford et al.，2021），一个跨模态模型预训练的400 M图像+字幕对从网络上，可以用于鲁棒的自动评估图像描述，而不需要参考。跨越几个语料库的实验表明，我们的新的无参考指标，CLIPScore，实现了最高的相关性与人类的判断，优于现有的基于参考的指标，如CIDER和SPICE。信息增益实验表明，CLIPScore，其紧密关注的图像-文本兼容性，是补充现有的基于参考的指标，强调文本-文本的相似性。因此，我们还提出了一个参考增强版本，RefCLIPScore，它实现了更高的相关性。除了文字描述任务之外，几个案例研究揭示了CLIPScore表现良好的领域（剪贴画图像，替代文本评级），但与基于参考的指标相比，它也相对较弱，例如，新闻标题需要更丰富的上下文知识。
Microsoft Research（微软研究院）；University of Washington（华盛顿大学）；University of California at Berkeley（加州大学伯克利分校）；IDSIA（IDSIA研究所）；USI-SUPSI（意大利语大学-瑞士南部应用科学与艺术大学）；Facebook AI Research（脸书人工智能研究院）；Google（谷歌）
