语言大模型 / LLM

Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmark

Elizabeth Fons, Rachneet Kaur, Soham Palande, Zhen Zeng, Tucker Balch, Manuela Veloso, Svitlana Vyetrenko

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted to EMNLP 2024

Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works

Xinfeng Yuan, Siyu Yuan, Yuhan Cui, Tianhe Lin, Xintao Wang, Rui Xu, Jiangjie Chen, Deqing Yang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments EMNLP 2024 camera-ready

Investigating large language models for their competence in extracting grammatically sound sentences from transcribed noisy utterances

Alina Wróblewska

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted at CoNLL 2024

PersonalSum: A User-Subjective Guided Personalized Summarization Dataset for Large Language Models

Lemei Zhang, Peng Liu, Marcus Tiedemann Oekland Henriksboe, Even W. Lauvrak, Jon Atle Gulla, Heri Ramampiaro

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted at NeurIPS 2024 Track on Datasets and Benchmarks. Code available at https://github.com/SmartmediaAI/PersonalSum

Defining Knowledge: Bridging Epistemology and Large Language Models

Constanza Fierro, Ruchira Dhar, Filippos Stamatiou, Nicolas Garneau, Anders Søgaard

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments EMNLP 2024

Quantifying Generalization Complexity for Large Language Models

Zhenting Qi, Hongyin Luo, Xuliang Huang, Zhuokai Zhao, Yibo Jiang, Xiangjun Fan, Himabindu Lakkaraju, James Glass

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Generalists vs. Specialists: Evaluating Large Language Models for Urdu

Samee Arif, Abdul Hameed Azeemi, Agha Ali Raza, Awais Athar

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models

Bolei Ma, Xinpeng Wang, Tiancheng Hu, Anna-Carolina Haensch, Michael A. Hedderich, Barbara Plank, Frauke Kreuter

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments EMNLP 2024 Findings

Enhanced Automated Code Vulnerability Repair using Large Language Models

David de-Fitero-Dominguez, Eva Garcia-Lopez, Antonio Garcia-Cabot, Jose-Javier Martinez-Herraiz

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Journal ref Engineering Applications of Artificial Intelligence. Volume 138, Part A, December 2024, 109291

Unveiling the Achilles' Heel of NLG Evaluators: A Unified Adversarial Framework Driven by Large Language Models

Yiming Chen, Chen Zhang, Danqing Luo, Luis Fernando D'Haro, Robby T. Tan, Haizhou Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments ACL24 Findings

A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery

Yu Zhang, Xiusi Chen, Bowen Jin, Sheng Wang, Shuiwang Ji, Wei Wang, Jiawei Han

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 35 pages; Accepted to EMNLP 2024 (Project Page: https://github.com/yuzhimanhua/Awesome-Scientific-Language-Models)

MultiPragEval: Multilingual Pragmatic Evaluation of Large Language Models

Dojun Park, Jiwoo Lee, Seohyun Park, Hyeyun Jeong, Youngeun Koo, Soonha Hwang, Seonwoo Park, Sungeun Lee

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments The 2nd GenBench workshop on generalisation (benchmarking) in NLP

TrustLLM: Trustworthiness in Large Language Models

Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bertie Vidgen, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric Xing, Furong Huang, Hao Liu, Heng Ji, Hongyi Wang, Huan Zhang, Huaxiu Yao, Manolis Kellis, Marinka Zitnik, Meng Jiang, Mohit Bansal, James Zou, Jian Pei, Jian Liu, Jianfeng Gao, Jiawei Han, Jieyu Zhao, Jiliang Tang, Jindong Wang, Joaquin Vanschoren, John Mitchell, Kai Shu, Kaidi Xu, Kai-Wei Chang, Lifang He, Lifu Huang, Michael Backes, Neil Zhenqiang Gong, Philip S. Yu, Pin-Yu Chen, Quanquan Gu, Ran Xu, Rex Ying, Shuiwang Ji, Suman Jana, Tianlong Chen, Tianming Liu, Tianyi Zhou, William Wang, Xiang Li, Xiangliang Zhang, Xiao Wang, Xing Xie, Xun Chen, Xuyu Wang, Yan Liu, Yanfang Ye, Yinzhi Cao, Yong Chen, Yue Zhao

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments This work is still under work and we welcome your contribution

Privacy in Large Language Models: Attacks, Defenses and Future Directions

Haoran Li, Yulin Chen, Jinglong Luo, Jiecong Wang, Hao Peng, Yan Kang, Xiaojin Zhang, Qi Hu, Chunkit Chan, Zenglin Xu, Bryan Hooi, Yangqiu Song

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments We upload the survey to cover more recent papers and inlcude privacy resaearch on multi-modality

CJEval: A Benchmark for Assessing Large Language Models Using Chinese Junior High School Exam Data

Qian-Wen Zhang, Haochen Wang, Fang Li, Siyu An, Lingfeng Qiao, Liangcai Gao, Di Yin, Xing Sun

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

RMCBench: Benchmarking Large Language Models' Resistance to Malicious Code

Jiachi Chen, Qingyuan Zhong, Yanlin Wang, Kaiwen Ning, Yongkun Liu, Zenan Xu, Zhe Zhao, Ting Chen, Zibin Zheng

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 12 pages, 6 figures, 5 tables, 39th IEEE/ACM International Conference on Automated Software Engineering (ASE '24)

Speech-Copilot: Leveraging Large Language Models for Speech Processing via Task Decomposition, Modularization, and Program Generation

Chun-Yi Kuan, Chih-Kai Yang, Wei-Ping Huang, Ke-Han Lu, Hung-yi Lee

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted to SLT 2024

Evaluating the Quality of Code Comments Generated by Large Language Models for Novice Programmers

Aysa Xuemo Fan, Arun Balajiee Lekshmi Narayanan, Mohammad Hassany, Jiaze Ke

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

The use of GPT-4o and Other Large Language Models for the Improvement and Design of Self-Assessment Scales for Measurement of Interpersonal Communication Skills

Goran Bubaš

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 41 pages

Graph Learning and Its Advancements on Large Language Models: A Holistic Survey

Shaopeng Wei, Jun Wang, Yu Zhao, Xingyan Chen, Qing Li, Fuzhen Zhuang, Ji Liu, Fuji Ren, Gang Kou

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 23 pages, 8 figures, 4 tables

JMedBench: A Benchmark for Evaluating Japanese Biomedical Large Language Models

Junfeng Jiang, Jiahao Huang, Akiko Aizawa

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Development and bilingual evaluation of Japanese medical large language model within reasonably low computational resources

Issey Sukeda

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 18 pages, 9 tables

Assessing Student Errors in Experimentation Using Artificial Intelligence and Large Language Models: A Comparative Study with Human Raters

Arne Bewersdorff, Kathrin Seßler, Armin Baur, Enkelejda Kasneci, Claudia Nerdel

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Enriching Datasets with Demographics through Large Language Models: What's in a Name?

Khaled AlNuaimi, Gautier Marti, Mathieu Ravaut, Abdulla AlKetbi, Andreas Henschel, Raed Jaradat

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 8 pages, 7 Tables, 5 Figures

Internal Consistency and Self-Feedback in Large Language Models: A Survey

Xun Liang, Shichao Song, Zifan Zheng, Hanyu Wang, Qingchen Yu, Xunkai Li, Rong-Hua Li, Yi Wang, Zhonghao Wang, Feiyu Xiong, Zhiyu Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 20 pages, 10 figures, 6 tables, 13 equations

Agentic Society: Merging skeleton from real world and texture from Large Language Model

Yuqi Bai, Kun Sun, Huishi Yin

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 16 pages, 5 figures and 4 tables

OrthoDoc: Multimodal Large Language Model for Assisting Diagnosis in Computed Tomography

Youzhu Jin, Yichen Zhang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments 8 pages, 1 figure

Steering Conversational Large Language Models for Long Emotional Support Conversations

Navid Madani, Sougata Saha, Rohini Srihari

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?

Yuwei Zhao, Ziyang Luo, Yuchen Tian, Hongzhan Lin, Weixiang Yan, Annan Li, Jing Ma

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments The first two authors contributed equally

An Empirical Study on Information Extraction using Large Language Models

Ridong Han, Chaohao Yang, Tao Peng, Prayag Tiwari, Xiang Wan, Lu Liu, Benyou Wang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 52 pages, Version 2.0; This article has an original arxiv version entitled "Is Information Extraction Solved by ChatGPT? An Analysis of Performance, Evaluation Criteria, Robustness and Errors'', whose url link is arXiv:2305.14450v1