代码大模型 / AI 编程

Themisto: Jupyter-Based Runtime Benchmark

Konstantin Grotov, Sergey Titov

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI、cs.LG

Comments Accepted to the third Deep Learning for Code (DL4C) workshop @ ICLR 2025

VeriMind: Agentic LLM for Automated Verilog Generation with a Novel Evaluation Metric

Bardia Nadimi, Ghali Omar Boutaib, Hao Zheng

专题命中 代码评测 ：code generation(abstract)；分类 cs.AI、cs.LG、cs.PL

A System for Comprehensive Assessment of RAG Frameworks

Mattia Rengo, Senad Beadini, Domenico Alfano, Roberto Abbruzzese

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Technical Report, 7 pages, 2 figures, 1 table

ResBench: Benchmarking LLM-Generated FPGA Designs with Resource Awareness

Ce Guo, Tong Zhao

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments to be published in International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies 2025

The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination

Yifan Sun, Han Wang, Dongbai Li, Gang Wang, Huan Zhang

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 23 pages

Can LLMs Enable Verification in Mainstream Programming?

Aleksandr Shefer, Igor Engel, Stanislav Alekseev, Daniil Berezun, Ekaterina Verbitskaia, Anton Podkopaev

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI、cs.PL

Automatically Labeling Clinical Trial Outcomes: A Large-Scale Benchmark for Drug Development

Chufan Gao, Jathurshan Pradeepkumar, Trisha Das, Shivashankar Thati, Jimeng Sun

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

CS-Bench: A Comprehensive Benchmark for Large Language Models towards Computer Science Mastery

Xiaoshuai Song, Muxi Diao, Guanting Dong, Zhengyang Wang, Yujia Fu, Runqi Qiao, Zhexu Wang, Dayuan Fu, Huangxuan Wu, Bin Liang, Weihao Zeng, Yejie Wang, Zhuoma GongQue, Jianing Yu, Qiuna Tan, Weiran Xu

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ICLR 2025

Scattered Forest Search: Smarter Code Space Exploration with LLMs

Jonathan Light, Yue Wu, Yiyou Sun, Wenchao Yu, Yanchi liu, Xujiang Zhao, Ziniu Hu, Haifeng Chen, Wei Cheng

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI、cs.LG

Comments Accepted at ICLR 2025 Conference

DataSciBench: An LLM Agent Benchmark for Data Science

Dan Zhang, Sining Zhoubian, Min Cai, Fengzu Li, Lekang Yang, Wei Wang, Tianjiao Dong, Ziniu Hu, Jie Tang, Yisong Yue

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 40 pages, 7 figures, 6 tables

How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmark

Ruizhong Qiu, Weiliang Will Zeng, James Ezick, Christopher Lott, Hanghang Tong

专题命中 代码评测 ：program synthesis(abstract)；分类 cs.SE、cs.AI、cs.LG

Comments ICLR 2025

CSR-Bench: Benchmarking LLM Agents in Deployment of Computer Science Research Repositories

Yijia Xiao, Runhui Wang, Luyang Kong, Davor Golac, Wei Wang

专题命中 代码评测 ：repository(abstract)；分类 cs.SE、cs.AI、cs.LG

Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey

Liang Chen, Zekun Wang, Shuhuai Ren, Lei Li, Haozhe Zhao, Yunshui Li, Zefan Cai, Hongcheng Guo, Lei Zhang, Yizhe Xiong, Yichi Zhang, Ruoyu Wu, Qingxiu Dong, Ge Zhang, Jian Yang, Lingwei Meng, Shujie Hu, Yulong Chen, Junyang Lin, Shuai Bai, Andreas Vlachos, Xu Tan, Minjia Zhang, Wen Xiao, Aaron Yee, Tianyu Liu, Baobao Chang

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 69 papes, 18 figures, repo at https://github.com/LMM101/Awesome-Multimodal-Next-Token-Prediction

ChainStream: An LLM-based Framework for Unified Synthetic Sensing

Jiacheng Liu, Yuanchun Li, Liangyan Li, Yi Sun, Hao Wen, Xiangyu Li, Yao Guo, Yunxin Liu

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments 18 pages, 8 figures

InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct

Yutong Wu, Di Huang, Wenxuan Shi, Wei Wang, Lingzhe Gao, Shihao Liu, Ziyuan Nan, Kaizhao Yuan, Rui Zhang, Xishan Zhang, Zidong Du, Qi Guo, Yewen Pu, Dawei Yin, Xing Hu, Yunji Chen

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments Accepted for publication at AAAI 2025. Extended version with full appendix, 18 pages

Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases

Zian Su, Xiangzhe Xu, Ziyang Huang, Kaiyuan Zhang, Xiangyu Zhang

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.CL、cs.AI

Open-Source Conversational AI with SpeechBrain 1.0

Mirco Ravanelli, Titouan Parcollet, Adel Moumen, Sylvain de Langen, Cem Subakan, Peter Plantinga, Yingzhi Wang, Pooneh Mousavi, Luca Della Libera, Artem Ploujnikov, Francesco Paissan, Davide Borra, Salah Zaiem, Zeyu Zhao, Shucong Zhang, Georgios Karakasidis, Sung-Lin Yeh, Pierre Champion, Aku Rouhe, Rudolf Braun, Florian Mai, Juan Zuluaga-Gomez, Seyed Mahed Mousavi, Andreas Nautsch, Ha Nguyen, Xuechen Liu, Sangeet Sagar, Jarod Duret, Salima Mdhaffar, Gaelle Laperriere, Mickael Rouvier, Renato De Mori, Yannick Esteve

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to the Journal of Machine Learning research (JMLR), Machine Learning Open Source Software

Deep Configuration Performance Learning: A Systematic Survey and Taxonomy

Jingzhi Gong, Tao Chen

专题命中 代码评测 ：repository(abstract)；分类 cs.SE、cs.AI、cs.LG

Comments Accepted by the TOSEM survey track in October 2024

Learning Code Preference via Synthetic Evolution

Jiawei Liu, Thanh Nguyen, Mingyue Shang, Hantian Ding, Xiaopeng Li, Yu Yu, Varun Kumar, Zijian Wang

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.LG

CELI: Controller-Embedded Language Model Interactions

Jan-Samuel Wagner, Dave DeCaprio, Abishek Chiffon Muthu Raja, Jonathan M. Holman, Lauren K. Brady, Sky C. Cheung, Hosein Barzekar, Eric Yang, Mark Anthony Martinez, David Soong, Sriram Sridhar, Han Si, Brandon W. Higgs, Hisham Hamadeh, Scott Ogden

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments 26 pages, 2 figures

Representation Learning of Structured Data for Medical Foundation Models

Vijay Prakash Dwivedi, Viktor Schlegel, Andy T. Liu, Thanh-Tung Nguyen, Abhinav Ramesh Kashyap, Jeng Wei, Wei-Hsian Yin, Stefan Winkler, Robby T. Tan

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2024 Workshop on Unifying Representations in Neural Models (UniReps 2024)

RGD: Multi-LLM Based Agent Debugger via Refinement and Generation Guidance

Haolin Jin, Zechao Sun, Huaming Chen

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.AI

Narrow Transformer: StarCoder-Based Java-LM For Desktop

Kamalkumar Rathinasamy, Balaji A J, Ankush Kumar, Gagan Gayari, Harshini K, Rajab Ali Mondal, Sreenivasa Raghavan K S, Swayam Singh, Mohammed Rafee Tarafdar

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments Updated Authors list

Agentic Retrieval-Augmented Generation for Time Series Analysis

Chidaksh Ravuru, Sagar Srinivas Sakhinana, Venkataramana Runkana

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Paper was accepted for Undergraduate Consortium at ACM KDD, 2024. Please find the link: https://kdd2024.kdd.org/undergraduate-consortium/

From Feature Importance to Natural Language Explanations Using LLMs with RAG

Sule Tekkesinoglu, Lars Kunze

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI、cs.LG

Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation

Tu Vu, Kalpesh Krishna, Salaheddin Alzubi, Chris Tar, Manaal Faruqui, Yun-Hsuan Sung

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 31 pages, 5 figures, 7 tables

ORAN-Bench-13K: An Open Source Benchmark for Assessing LLMs in Open Radio Access Networks

Pranshav Gajjar, Vijay K. Shah

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI、cs.LG

DiscoveryBench: Towards Data-Driven Discovery with Large Language Models

Bodhisattwa Prasad Majumder, Harshit Surana, Dhruv Agarwal, Bhavana Dalvi Mishra, Abhijeetsingh Meena, Aryan Prakhar, Tirth Vora, Tushar Khot, Ashish Sabharwal, Peter Clark

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Website: https://github.com/allenai/discoverybench

Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency

Baizhou Huang, Shuai Lu, Weizhu Chen, Xiaojun Wan, Nan Duan

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments ACL 2024

Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code

Ziyin Zhang, Chaoyu Chen, Bingchang Liu, Cong Liao, Zi Gong, Hang Yu, Jianguo Li, Rui Wang

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.CL、cs.AI

Comments Repo: https://github.com/codefuse-ai/Awesome-Code-LLM. 9 figures, 18 tables, and 902 references. Under review