语言大模型 / LLM

Teaching Large Language Models to Self-Debug

Xinyun Chen, Maxwell Lin, Nathanael Schärli, Denny Zhou

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

In-context Interference in Chat-based Large Language Models

Eric Nuertey Coleman, Julio Hurtado, Vincenzo Lomonaco

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Can large language models generate salient negative statements?

Hiba Arnaout, Simon Razniewski

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments For data, see https://www.mpi-inf.mpg.de/fileadmin/inf/d5/research/negation_in_KBs/data.csv

An Empirical Study of NetOps Capability of Pre-Trained Large Language Models

Yukai Miao, Yu Bai, Li Chen, Dan Li, Haifeng Sun, Xizheng Wang, Ziqiu Luo, Yanyu Ren, Dapeng Sun, Xiuting Xu, Qi Zhang, Chao Xiang, Xinchi Li

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Chinese Fine-Grained Financial Sentiment Analysis with Large Language Models

Yinyu Lan, Yanru Wu, Wang Xu, Weiqiang Feng, Youhao Zhang

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by (FinLLM 2023)@IJCAI 2023, https://finllm.github.io/workshop/#/fcb

AGIBench: A Multi-granularity, Multimodal, Human-referenced, Auto-scoring Benchmark for Large Language Models

Fei Tang, Wanling Gao, Luzhou Peng, Jianfeng Zhan

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 14 pages

Diminished Diversity-of-Thought in a Standard Large Language Model

Peter S. Park, Philipp Schoenegger, Chongyang Zhu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 67 pages (42-page main text, 25-page SI); 12 visualizations (four tables and three figures in the main text, five figures in the SI); additional exploratory follow-up study varied the demographic details preceding the prompt; preregistered OSF database is available at https://osf.io/dzp8t/

Large Language Models are not Fair Evaluators

Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, Zhifang Sui

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

AI in the Gray: Exploring Moderation Policies in Dialogic Large Language Models vs. Human Answers in Controversial Topics

Vahid Ghafouri, Vibhor Agarwal, Yong Zhang, Nishanth Sastry, Jose Such, Guillermo Suarez-Tangil

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation

Xiaowei Huang, Wenjie Ruan, Wei Huang, Gaojie Jin, Yi Dong, Changshun Wu, Saddek Bensalem, Ronghui Mu, Yi Qi, Xingyu Zhao, Kaiwen Cai, Yanghao Zhang, Sihao Wu, Peipei Xu, Dengyu Wu, Andre Freitas, Mustafa A. Mustafa

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

Can ChatGPT Detect Intent? Evaluating Large Language Models for Spoken Language Understanding

Mutian He, Philip N. Garner

专题命中 评测与基准 ：language model(title,abstract)；large language model(title)；prompting(abstract)；分类 cs.CL、cs.AI

Comments 6 pages, 2 figures; Accepted by Interspeech 2023

The Costly Dilemma: Generalization, Evaluation and Cost-Optimal Deployment of Large Language Models

Abi Aryan, Aakash Kumar Nain, Andrew McMahon, Lucas Augusto Meyer, Harpreet Singh Sahota

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments 11 pages

A Critical Review of Large Language Models: Sensitivity, Bias, and the Path Toward Specialized AI

Arash Hajikhani, Carolyn Cole

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 6 figures, 6 tables

On the application of Large Language Models for language teaching and assessment technology

Andrew Caines, Luca Benedetto, Shiva Taslimipoor, Christopher Davis, Yuan Gao, Oeistein Andersen, Zheng Yuan, Mark Elliott, Russell Moore, Christopher Bryant, Marek Rei, Helen Yannakoudakis, Andrew Mullooly, Diane Nicholls, Paula Buttery

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted at the AIED2023 workshop: Empowering Education with LLMs - the Next-Gen Interface and Content Generation

Can Large Language Models Empower Molecular Property Prediction?

Chen Qian, Huayi Tang, Zhirui Yang, Hong Liang, Yong Liu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

INT-FP-QSim: Mixed Precision and Formats For Large Language Models and Vision Transformers

Lakshmi Nair, Mikhail Bernadskiy, Arulselvan Madhavan, Craig Chan, Ayon Basumallik, Darius Bunandar

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments This report is supplementary material to the open-source code available at: https://github.com/lightmatter-ai/INT-FP-QSim

CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Models

Yufei Huang, Deyi Xiong

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

TrustGPT: A Benchmark for Trustworthy and Responsible Large Language Models

Yue Huang, Qihui Zhang, Philip S. Y, Lichao Sun

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments We are currently expanding this work and welcome collaborators!

Sensitivity and Robustness of Large Language Models to Prompt Template in Japanese Text Classification Tasks

Chengguang Gan, Tatsunori Mori

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Under Review. 11 pages, 8 figures

The Two Word Test: A Semantic Benchmark for Large Language Models

Nicholas Riccardi, Rutvik H. Desai

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 12 pages, 5 figures, 3 tables, submitted to NeurIPS 2023 Datasets and Benchmarks Track

ANALOGICAL -- A Novel Benchmark for Long Text Analogy Evaluation in Large Language Models

Thilini Wijesiriwardene, Ruwan Wickramarachchi, Bimal G. Gajera, Shreeyash Mukul Gowaikar, Chandan Gupta, Aman Chadha, Aishwarya Naresh Reganti, Amit Sheth, Amitava Das

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted as a long paper at Findings of ACL 2023

EvEval: A Comprehensive Evaluation of Event Semantics for Large Language Models

Zhengwei Tao, Zhi Jin, Xiaoying Bai, Haiyan Zhao, Yanlin Feng, Jia Li, Wenpeng Hu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Have Large Language Models Developed a Personality?: Applicability of Self-Assessment Tests in Measuring Personality in LLMs

Xiaoyang Song, Akshat Gupta, Kiyan Mohebbizadeh, Shujie Hu, Anant Singh

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Automated Code generation for Information Technology Tasks in YAML through Large Language Models

Saurabh Pujar, Luca Buratti, Xiaojie Guo, Nicolas Dupuis, Burn Lewis, Sahil Suneja, Atin Sood, Ganesh Nalawade, Matthew Jones, Alessandro Morari, Ruchir Puri

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Are Emergent Abilities of Large Language Models a Mirage?

Rylan Schaeffer, Brando Miranda, Sanmi Koyejo

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

Understanding HTML with Large Language Models

Izzeddin Gur, Ofir Nachum, Yingjie Miao, Mustafa Safdari, Austin Huang, Aakanksha Chowdhery, Sharan Narang, Noah Fiedel, Aleksandra Faust

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

Surfacing Biases in Large Language Models using Contrastive Input Decoding

Gal Yona, Or Honovich, Itay Laish, Roee Aharoni

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Professional Certification Benchmark Dataset: The First 500 Jobs For Large Language Models

David Noever, Matt Ciolino

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

Large Language Models Meet NL2Code: A Survey

Daoguang Zan, Bei Chen, Fengji Zhang, Dianjie Lu, Bingchao Wu, Bei Guan, Yongji Wang, Jian-Guang Lou

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to the main conference of ACL 2023 (long paper)

Shortcut Learning of Large Language Models in Natural Language Understanding

Mengnan Du, Fengxiang He, Na Zou, Dacheng Tao, Xia Hu

专题命中 评测与基准 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted by Communications of the ACM (CACM), Review Article