MultiZoo & MultiBench：面向多模态深度学习的标准化工具包
MultiZoo & MultiBench: A Standardized Toolkit for Multimodal Deep Learning
AI总结：
本文发布了包含>20个算法的多模态工具包MultiZoo和涵盖15个数据集的大规模基准MultiBench，提供自动化端到端流水线与综合评估方法，推动多模态模型的易用性与可复现性。
摘要：
学习多模态表征涉及整合来自多个异构数据源的信息。为了加速对研究不足的模态和任务的探索，同时确保现实世界的鲁棒性，我们发布了MultiZoo，这是一个包含>20个核心多模态算法标准化实现的公开工具包；同时发布了MultiBench，这是一个涵盖15个数据集、10种模态、20个预测任务和6个研究领域的大规模基准。两者结合，提供了一条自动化的端到端机器学习流水线，简化并标准化了数据加载、实验设置和模型评估。为实现全面评估，我们提供了一套综合方法论来评估(1)泛化能力、(2)时间和空间复杂度以及(3)模态鲁棒性。MultiBench为更好地理解多模态模型的能力与局限铺平了道路，同时确保了易用性、可访问性和可复现性。我们的工具包已公开发布，将定期更新，并欢迎社区贡献。
英文摘要：
Learning multimodal representations involves integrating information from multiple heterogeneous sources of data. In order to accelerate progress towards understudied modalities and tasks while ensuring real-world robustness, we release MultiZoo, a public toolkit consisting of standardized implementations of > 20 core multimodal algorithms and MultiBench, a large-scale benchmark spanning 15 datasets, 10 modalities, 20 prediction tasks, and 6 research areas. Together, these provide an automated end-to-end machine learning pipeline that simplifies and standardizes data loading, experimental setup, and model evaluation. To enable holistic evaluation, we offer a comprehensive methodology to assess (1) generalization, (2) time and space complexity, and (3) modality robustness. MultiBench paves the way towards a better understanding of the capabilities and limitations of multimodal models, while ensuring ease of use, accessibility, and reproducibility. Our toolkits are publicly available, will be regularly updated, and welcome inputs from the community.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
