代码大模型 / AI 编程

Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs

Sukmin Yun, Haokun Lin, Rusiru Thushara, Mohammad Qazim Bhat, Yongxin Wang, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, Timothy Baldwin, Zhengzhong Liu, Eric P. Xing, Xiaodan Liang, Zhiqiang Shen

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024 Datasets and Benchmarks Camera-ready Version. Website at https://mbzuai-llm.github.io/webpage2code/

InfiBench: Evaluating the Question-Answering Capabilities of Code Large Language Models

Linyi Li, Shijie Geng, Zhenwen Li, Yibo He, Hao Yu, Ziyue Hua, Guanghan Ning, Siwei Wang, Tao Xie, Hongxia Yang

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.LG

Comments 31 pages. Appear at NeurIPS 2024 Datasets and Benchmarks track. Project website: https://infi-coder.github.io/infibench

Synthesize, Partition, then Adapt: Eliciting Diverse Samples from Foundation Models

Yeming Wen, Swarat Chaudhuri

专题命中 代码评测 ：code generation(abstract)；分类 cs.AI、cs.LG

Tool Learning with Large Language Models: A Survey

Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, Ji-Rong Wen

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments The article has been accepted by Frontiers of Computer Science (FCS), with the DOI: {10.1007/s11704-024-40678-2}

GRAMMAR: Grounded and Modular Methodology for Assessment of Closed-Domain Retrieval-Augmented Language Model

Xinzhe Li, Ming Liu, Shang Gao

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments Under Review

Learning Language Structures through Grounding

Freda Shi

专题命中 代码评测 ：program synthesis(abstract)；分类 cs.CL、cs.AI

Comments Ph.D. Thesis

User-centric evaluation of explainability of AI with and for humans: a comprehensive empirical study

Szymon Bobek, Paloma Korycińska, Monika Krakowska, Maciej Mozolewski, Dorota Rak, Magdalena Zych, Magdalena Wójcik, Grzegorz J. Nalepa

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmers

Hussein Mozannar, Valerie Chen, Mohammed Alsobay, Subhro Das, Sebastian Zhao, Dennis Wei, Manish Nagireddy, Prasanna Sattigeri, Ameet Talwalkar, David Sontag

专题命中 代码评测 ：code model(abstract)；分类 cs.SE、cs.AI

Apollo: A Lightweight Multilingual Medical LLM towards Democratizing Medical AI to 6B People

Xidong Wang, Nuo Chen, Junyin Chen, Yidong Wang, Guorui Zhen, Chunxian Zhang, Xiangbo Wu, Yan Hu, Anningzhe Gao, Xiang Wan, Haizhou Li, Benyou Wang

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments Preprint

Mark My Words: Analyzing and Evaluating Language Model Watermarks

Julien Piet, Chawin Sitawarin, Vivian Fang, Norman Mu, David Wagner

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI

Comments 22 pages, 18 figures

MOMENT: A Family of Open Time-series Foundation Models

Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, Artur Dubrawski

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

Comments Accepted at ICML'24. This is a revision. See changelog in the Appendix

GEMS: Generative Expert Metric System through Iterative Prompt Priming

Ti-Chung Cheng, Carmen Badea, Christian Bird, Thomas Zimmermann, Robert DeLine, Nicole Forsgren, Denae Ford

专题命中 代码评测 ：repository(abstract)；分类 cs.SE、cs.AI

Comments 29 pages, 3 figures

BabelBench: An Omni Benchmark for Code-Driven Analysis of Multimodal and Multistructured Data

Xuwu Wang, Qiwen Cui, Yunzhe Tao, Yiran Wang, Ziwei Chai, Xiaotian Han, Boyi Liu, Jianbo Yuan, Jing Su, Guoyin Wang, Tingkai Liu, Liyu Chen, Tianyi Liu, Tao Sun, Yufeng Zhang, Sirui Zheng, Quanzeng You, Yang Yang, Hongxia Yang

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI

CodeInsight: A Curated Dataset of Practical Coding Solutions from Stack Overflow

Nathanaël Beau, Benoît Crabbé

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL

Comments Accepted to ACL 2024 Findings

A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations

Jinqiang Wang, Huansheng Ning, Yi Peng, Qikai Wei, Daniel Tesfai, Wenwei Mao, Tao Zhu, Runhe Huang

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments 25 pages,4 figures

Transformer with Controlled Attention for Synchronous Motion Captioning

Karim Radouane, Sylvie Ranwez, Julien Lagarde, Andon Tchechmedjiev

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.LG

CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?

Yuwei Zhao, Ziyang Luo, Yuchen Tian, Hongzhan Lin, Weixiang Yan, Annan Li, Jing Ma

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.CL

Comments The first two authors contributed equally

A Survey on Benchmarks of Multimodal Large Language Models

Jian Li, Weiheng Lu, Hao Fei, Meng Luo, Ming Dai, Min Xia, Yizhang Jin, Zhenye Gan, Ding Qi, Chaoyou Fu, Ying Tai, Wankou Yang, Yabiao Wang, Chengjie Wang

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

SALLM: Security Assessment of Generated Code

Mohammed Latif Siddiq, Joanna C. S. Santos, Sajith Devareddy, Anna Muller

专题命中 代码评测 ：code generation(abstract)；分类 cs.SE、cs.AI

Comments Accepted at the 6th International Workshop on Automated and verifiable Software sYstem DEvelopment (ASYDE) with ASE Conference 2024

Journal ref 39th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW '24), October 27-November 1, 2024, Sacramento, CA, USA, ACM, New York, NY, USA, 12 pages

Evaluating the Energy Consumption of Machine Learning: Systematic Literature Review and Experiments

Charlotte Rodriguez, Laura Degioanni, Laetitia Kameni, Richard Vidal, Giovanni Neglia

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

Comments 52 pages,

Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era

Sunhao Dai, Chen Xu, Shicheng Xu, Liang Pang, Zhenhua Dong, Jun Xu

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments KDD 2024 Tutorial&Survey; Tutorial Website: https://llm-ir-bias-fairness.github.io/

Which Side Are You On? A Multi-task Dataset for End-to-End Argument Summarisation and Evaluation

Hao Li, Yuping Wu, Viktor Schlegel, Riza Batista-Navarro, Tharindu Madusanka, Iqra Zahid, Jiayan Zeng, Xiaochi Wang, Xinran He, Yizhi Li, Goran Nenadic

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.LG

Comments Published on ACL 2024 Findings

CON-FOLD -- Explainable Machine Learning with Confidence

Lachlan McGinness, Peter Baumgartner

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

rule4ml: An Open-Source Tool for Resource Utilization and Latency Estimation for ML Models on FPGA

Mohammad Mehdi Rahimifar, Hamza Ezzaoui Rahali, Audrey C. Therrien

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

SafeDreamer: Safe Reinforcement Learning with World Models

Weidong Huang, Jiaming Ji, Chunhe Xia, Borong Zhang, Yaodong Yang

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

Comments ICLR 2024

PyBench: Evaluating LLM Agent on various real-world coding tasks

Yaolun Zhang, Yinxu Pan, Yudong Wang, Jie Cai

专题命中 代码评测 ：repository(abstract)；分类 cs.SE、cs.AI

Comments 16 pages

LLMs Plagiarize: Ensuring Responsible Sourcing of Large Language Model Training Data Through Knowledge Graph Comparison

Devam Mondal, Carlo Lipizzi

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.LG

Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs

Andries Smit, Paul Duckworth, Nathan Grinsztajn, Thomas D. Barrett, Arnu Pretorius

专题命中 代码评测 ：repository(abstract)；分类 cs.CL、cs.AI

Comments 2 pages, 13 figures

SafePowerGraph: Safety-aware Evaluation of Graph Neural Networks for Transmission Power Grids

Salah Ghamizi, Aleksandar Bojchevski, Aoxiang Ma, Jun Cao

专题命中 代码评测 ：repository(abstract)；分类 cs.AI、cs.LG

Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?

Ruisheng Cao, Fangyu Lei, Haoyuan Wu, Jixuan Chen, Yeqiao Fu, Hongcheng Gao, Xinzhuang Xiong, Hanchong Zhang, Yuchen Mao, Wenjing Hu, Tianbao Xie, Hongshen Xu, Danyang Zhang, Sida Wang, Ruoxi Sun, Pengcheng Yin, Caiming Xiong, Ansong Ni, Qian Liu, Victor Zhong, Lu Chen, Kai Yu, Tao Yu

专题命中 代码评测 ：code generation(abstract)；分类 cs.CL、cs.AI

Comments 34 pages, 14 figures, 10 tables