机构
*
Amazon Lab126, HW Tech Org.(亚马逊实验室126,硬件技术组织)
;
Computational Modeling and Simulation University of Pittsburgh(计算建模与仿真大学匹兹堡分校)
;
Mathematics & Statistics Department University of Minnesota Duluth(数学与统计学系明尼苏达大学 Duluth 分校)
ROG-Grasp: Root-Oriented Geometry for Robotic Grasping and Placement
ROG-Grasp:面向根部的几何方法用于机器人抓取与放置
Zijian An, Augustus Sroka, Ran Yang, Bill Cai, Satoru Eto, Brian Poon, Kelvin Cai, Shijie Geng, Feng Liu, Yiming Feng, Lifeng Zhou
机构
*
Department of Electrical and Computer Engineering, Drexel University(德雷塞尔大学电气与计算机工程系)
;
Virginia Seafood Agricultural Research and Extension Center, and Department of Biological Systems Engineering, Virginia Tech(弗吉尼亚理工学院生物系统工程系和弗吉尼亚海鲜农业研究与推广中心)
;
Amazon Store Foundation AI (SFAI)(亚马逊商店基金会人工智能(SFAI))
Lessons from the Trenches on Reproducible Evaluation of Language Models
关于语言模型可重复评估的前线经验教训
Stella Biderman, Hailey Schoelkopf, Lintang Sutawika, Leo Gao, Jonathan Tow, Baber Abbasi, Alham Fikri Aji, Pawan Sasanka Ammanamanchi, Sidney Black, Jordan Clive, Anthony DiPofi, Julen Etxaniz, Benjamin Fattori, Jessica Zosa Forde, Charles Foster, Jeffrey Hsu, Mimansa Jaiswal, Wilson Y. Lee, Haonan Li, Charles Lovering, Niklas Muennighoff, Ellie Pavlick, Jason Phang, Aviya Skowron, Samson Tan, Xiangru Tang, Kevin A. Wang, Genta Indra Winata, François Yvon, Andy Zou
机构
*
MBZUAI
;
IIIT Hyderabad
;
EleutherAI
;
HiTZ Center - Ixa, UPV/EHU
;
Ivy Natal
;
University of Michigan
;
HubSpot
;
LibrAI
;
Kensho
;
Contextual AI
;
Brown University
;
New York University
;
Amazon
;
Yale University
;
HKUST
;
Sorbonne University
;
CMU
AI总结
本文基于开发Language Model Evaluation Harness框架的三年经验,总结了语言模型评估中面临的方法论挑战、缺乏可重复性和透明度等问题,并提供了改进评估严谨性和信心的建议。