arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 1902.03570cs.AIcs.CLcs.CVcs.LG
收藏

EvalAI:为人工智能智能体建立更好的评估系统

EvalAI: Towards Better Evaluation Systems for AI Agents

Georgia Institute of Technology(佐治亚理工学院);Paytm;Zomato(印度餐食配送平台(Zomato));Cyware

Deshraj Yadav, Rishabh Jain, Harsh Agrawal, Prithvijit Chattopadhyay, Taranjeet Singh, Akash Jain, Shiv Baran Singh, Stefan Lee, Dhruv Batra

摘要:

我们介绍EvalAI,这是一个开源平台,用于评估和比较机器学习(ML)和人工智能(AI)算法。EvalAI旨在为研究社区提供一个可扩展的解决方案,以满足评估机器学习模型和代理在环境中对注释或与人类互动的关键需求。这将帮助研究人员、学生和数据科学家创建、协作和参与全球组织的人工智能挑战。通过简化和标准化这些模型的基准测试过程,EvalAI寻求降低参与全球科学努力的门槛,以推动机器学习和人工智能的前沿,从而提高该领域可衡量的进展速度。

英文摘要:

We introduce EvalAI, an open source platform for evaluating and comparing machine learning (ML) and artificial intelligence algorithms (AI) at scale. EvalAI is built to provide a scalable solution to the research community to fulfill the critical need of evaluating machine learning models and agents acting in an environment against annotations or with a human-in-the-loop. This will help researchers, students, and data scientists to create, collaborate, and participate in AI challenges organized around the globe. By simplifying and standardizing the process of benchmarking these models, EvalAI seeks to lower the barrier to entry for participating in the global scientific effort to push the frontiers of machine learning and artificial intelligence, thereby increasing the rate of measurable progress in this domain.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑