Apollo：利用多专家进行零样本多模态推理
Apollo: Zero-shot MultiModal Reasoning with Multiple Experts
AI总结：
Apollo提出一种零样本多模态推理框架，通过多专家基础模型去中心化协作执行复杂任务，无需提示工程或定制训练，在风格化图像描述和音频感知图像描述任务上优于半监督最先进模型。
摘要：
我们提出一个模块化框架，利用不同基础模型在不同模态和领域上的专长，以执行单一、复杂的多模态任务，而无需依赖提示工程或专门定制的多模态训练。我们的方法支持去中心化的命令执行，并允许每个模型既贡献于其他模型的专长，也从中受益。由于不依赖提示，我们的方法可以扩展到多种基础模型（包括音频和视觉模型），而不仅限于语言模型。我们在两个任务上展示了该方法。在广为人知的风格化图像描述任务上，实验表明我们的方法优于半监督的最先进模型，同时保持零样本特性，并避免了昂贵的训练、数据收集和提示工程。我们进一步在一个新任务——音频感知图像描述上验证了该方法，该任务给定一张图像和一段音频，目标是生成在所提供的音频语境下描述该图像的文本。我们的代码已在GitHub上公开。
英文摘要：
We propose a modular framework that leverages the expertise of different foundation models over different modalities and domains in order to perform a single, complex, multi-modal task, without relying on prompt engineering or otherwise tailor-made multi-modal training. Our approach enables decentralized command execution and allows each model to both contribute and benefit from the expertise of the other models. Our method can be extended to a variety of foundation models (including audio and vision), above and beyond only language models, as it does not depend on prompts. We demonstrate our approach on two tasks. On the well-known task of stylized image captioning, our experiments show that our approach outperforms semi-supervised state-of-the-art models, while being zero-shot and avoiding costly training, data collection, and prompt engineering. We further demonstrate this method on a novel task, audio-aware image captioning, in which an image and audio are given and the task is to generate text that describes the image within the context of the provided audio. Our code is available on GitHub.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
