Autonomous program improvement typically involves automatically producing bug fixes and feature additions. Such program improvement can be accomplished by a combination of large language model (LLM) and program analysis capabilities, in the form of an LLM agent. Since program repair or program improvement typically requires a specification of intended behavior - specification inference can be useful for producing high quality program patches. In this work, we examine efficient and low-cost workflows for iterative specification inference within an LLM agent. Given a GitHub issue to be resolved in a software project, our goal is to conduct iterative code search accompanied by specification inference - thereby inferring intent from both the project structure and behavior. The intent thus captured is examined by a reviewer agent with the goal of vetting the patches as well as providing a measure of confidence in the vetted patches. Our approach SpecRover (AutoCodeRover-v2) is built on the open-source LLM agent AutoCodeRover. In an evaluation on the full SWE-Bench consisting of 2294 GitHub issues, it shows more than 50% improvement in efficacy over AutoCodeRover. Compared to the open-source agents available, our work shows modest cost ($0.65 per issue) in resolving an average GitHub issue in SWE-Bench lite. The production of explanation by SpecRover allows for a better "signal" to be given to the developer, on when the suggested patches can be accepted with confidence. SpecRover also seeks to demonstrate the continued importance of specification inference in automated program repair, even as program repair technologies enter the LLM era.
Fact in Fragments: Deconstructing Complex Claims via LLM-based Atomic Fact Extraction and Verification
Key Laboratory of Trustworthy Distributed Computing and Service (MoE)(可信分布式计算与服务重点实验室);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Jinan University(暨南大学);Department of Gastric Surgery, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital(Fujian Branch of Fudan University Shanghai Cancer Center)(福建医科大学胃外科临床肿瘤学院、福建肿瘤医院(复旦大学上海肿瘤中心福建分部))
Chat-Driven Optimal Management for Virtual Network Services
Department of Applied Physics and Physico-Informatics, Keio University(应用物理与物理信息学系,庆应大学);Network Service Laboratories, NTT, Inc.(网络服务实验室,NTT公司);IOWN Product Design Center, NTT, Inc.(IOWN产品设计中心,NTT公司)
Query-Driven Multimodal Information Extraction from Long Documents
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);Graduate School of Information Science and Technology, The University of Tokyo(东京大学信息科学与技术研究生院)
A Hybrid Supervised-LLM Pipeline for Actionable Suggestion Mining in Unstructured Customer Reviews
Department of Computer Science & Information Systems, Birla Institute of Technology and Science, Pilani, India(印度比拉理工学院计算机科学与信息系统系);Birdeye Inc.(Birdeye公司)
AAD-LLM: Neural Attention-Driven Auditory Scene Understanding
Department of Electrical Engineering(电气工程系);Department of Neurological Surgery(神经外科系);Mortimer B. Zuckerman Mind Brain Behavior Institute(莫提默·B·祖克曼心智-脑-行为研究所);Hofstra Northwell School of Medicine(霍夫曼斯特拉北well医学院);The Feinstein Institutes for Medical Research(费森特医学研究所);Neurology Department(神经病学系)
Demystifying the Privacy-Utility Trade-off in LLM Interactions
Soochow University(苏州大学);Beihang University(北京航空航天大学);Suzhou City University(苏州城市学院);Shanghai Key Lab of Intelligent Information Processing(上海市智能信息处理重点实验室);Shanghai AI Lab(上海人工智能实验室)
Conv-to-Bench: Evaluating Language Models Via User-Assistant Dialogues In Code Tasks
Institute of Mathematics and Computer Science, University of São Paulo(圣保罗大学数学与计算机科学学院);Institute of Informatics, Federal University of Goiás(戈亚斯联邦大学信息学院);HUG Labs(HUG实验室);Advanced Knowledge Center for Immersive Technologies (AKCIT)(沉浸式技术高级知识中心)