TroVE:归纳可验证且高效的工具箱以解决程序化任务
TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Tasks
AI总结:
本文提出无需训练的 TROVE 方法,让代码语言模型自动生成、扩充并精简可复用高级函数工具箱,在 11 个数学、表格问答和图像推理数据集上以更小工具箱获得更简单、更准确且更易人工验证的程序。
摘要:
语言模型(LMs)可以通过编写程序来解决诸如回答有关表格或图像的问题等任务。然而,使用原始函数往往会导致程序冗长且容易出错,而更高级的函数则需要专家设计。为了在无需人工劳动的情况下实现更好的解决方案,我们要求代码 LMs 整理可复用的高级函数,并使用它们编写解决方案。我们提出了 TROVE,这是一种无需训练的方法,通过“使用中生成”、扩充和定期精简工具箱来归纳出可验证且高效的函数工具箱。在来自数学、表格问答和图像推理任务的 11 个数据集上,与使用 CODELLAMA 的基线方法以及使用 GPT 的先前方法相比,TROVE 始终能产生更简单且准确率更高的解决方案,同时所用工具箱的规模小 79–98%。TROVE 还使人工验证比基线方法快 31%,准确率高 13%。借助同一流水线,它可为不同任务和数据集创建多样化函数,从而为理解它们各自的特征提供洞见。
英文摘要:
Language models (LMs) can solve tasks such as answering questions about tables or images by writing programs. However, using primitive functions often leads to verbose and error-prone programs, and higher-level functions require expert design. To enable better solutions without human labor, we ask code LMs to curate reusable high-level functions, and use them to write solutions. We present TROVE, a training-free method of inducing a verifiable and efficient toolbox of functions, by generating via using, growing, and periodically trimming the toolbox. On 11 datasets from math, table question answering, and image reasoning tasks, TROVE consistently yields simpler solutions with higher accuracy than baselines using CODELLAMA and previous methods using GPT, while using 79-98% smaller toolboxes. TROVE further enables 31% faster and 13% more accurate human verification than baselines. With the same pipeline, it creates diverse functions for varied tasks and datasets, providing insights into their individual characteristics.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。