ResBench：以资源感知为基准评测LLM生成的FPGA设计
ResBench: Benchmarking LLM-Generated FPGA Designs with Resource Awareness
AI总结：
针对现有LLM代码生成基准忽略FPGA硬件资源使用、多样性不足的问题，提出首个聚焦资源的基准ResBench，含12类56个问题，配套开源框架可自动评测，实验验证其能区分不同LLM生成HDL的资源优化水平。
摘要：
现场可编程门阵列（FPGA）在现代硬件设计中应用广泛，但为FPGA实现编写硬件描述语言（HDL）代码仍是一项复杂且耗时的任务。大语言模型（LLM）已成为HDL生成的潜力工具，但现有面向LLM的代码生成基准主要关注功能正确性，却忽略了硬件资源使用情况。此外，当前基准的多样性有限，无法全面代表各类真实世界FPGA应用。为弥补这些不足，我们提出ResBench——首个专为区分资源优化型与低效LLM生成HDL代码而设计的、聚焦资源的基准。ResBench包含12个类别共56个问题，覆盖从有限状态机到金融计算的各类应用。我们的开源评测框架通过生成Verilog代码、验证正确性并测量资源使用量，实现对LLM的自动测试。主要针对查找表（LUT）使用量的实验显示，不同LLM之间存在显著差异，证明ResBench能够识别生成更具资源优化性FPGA设计的模型。
英文摘要：
Field-Programmable Gate Arrays (FPGAs) are widely used in modern hardware design, yet writing Hardware Description Language (HDL) code for FPGA implementation remains a complex and time-consuming task. Large Language Models (LLMs) have emerged as a promising tool for HDL generation, but existing benchmarks for LLM-based code generation primarily focus on functional correctness while overlooking hardware resource usage. Furthermore, current benchmarks offer limited diversity and do not fully represent the wide range of real-world FPGA applications. To address these shortcomings, we introduce ResBench, the first resource-focused benchmark explicitly designed to distinguish between resource-optimized and inefficient LLM-generated HDL code. ResBench consists of 56 problems across 12 categories, covering applications from finite state machines to financial computing. Our open-source evaluation framework automatically tests LLMs by generating Verilog code, verifying correctness, and measuring resource usage. The experiments, which primarily analyze Lookup Table (LUT) usage, reveal significant differences among LLMs, demonstrating ResBench's capability to identify models that generate more resource-optimized FPGA designs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
