大模型推理能力

Editing Common Sense in Transformers

Anshita Gupta, Debanjan Mondal, Akshay Krishna Sheshadri, Wenlong Zhao, Xiang Lorraine Li, Sarah Wiegreffe, Niket Tandon

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to EMNLP 2023 Main Conference. Anshita, Debanjan, Akshay are co-first authors. Code and datasets for all experiments are available at https://github.com/anshitag/memit_csk

ExplainCPE: A Free-text Explanation Benchmark of Chinese Pharmacist Examination

Dongfang Li, Jindi Yu, Baotian Hu, Zhenran Xu, Min Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments EMNLP 2023 Findings

MUSER: A Multi-View Similar Case Retrieval Dataset

Qingquan Li, Yiran Hu, Feng Yao, Chaojun Xiao, Zhiyuan Liu, Maosong Sun, Weixing Shen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted by CIKM 2023 Resource Track

Journal ref CIKM 2023

Fighting Fire with Fire: The Dual Role of LLMs in Crafting and Detecting Elusive Disinformation

Jason Lucas, Adaku Uchendu, Michiharu Yamashita, Jooyoung Lee, Shaurya Rohatgi, Dongwon Lee

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted at EMNLP 2023

TableQAKit: A Comprehensive and Practical Toolkit for Table-based Question Answering

Fangyu Lei, Tongxu Luo, Pengqi Yang, Weihao Liu, Hanwen Liu, Jiahe Lei, Yiming Huang, Yifan Wei, Shizhu He, Jun Zhao, Kang Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Work in progress

ALCUNA: Large Language Models Meet New Knowledge

Xunjian Yin, Baizhou Huang, Xiaojun Wan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments EMNLP 2023

StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding

Cheng Jiayang, Lin Qiu, Tsz Ho Chan, Tianqing Fang, Weiqi Wang, Chunkit Chan, Dongyu Ru, Qipeng Guo, Hongming Zhang, Yangqiu Song, Yue Zhang, Zheng Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted by EMNLP 2023 main conference

Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations

Wei-Lin Chen, Cheng-Kuang Wu, Yun-Nung Chen, Hsin-Hsi Chen

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

Comments Accepted as a long paper at EMNLP 2023

Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU

Fajri Koto, Nurul Aisyah, Haonan Li, Timothy Baldwin

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted at EMNLP 2023

Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Dataset

Junling Liu, Peilin Zhou, Yining Hua, Dading Chong, Zhongyu Tian, Andrew Liu, Helin Wang, Chenyu You, Zhenhua Guo, Lei Zhu, Michael Lingzhi Li

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted by NeurIPS 2023 Datasets and Benchmarks Track

Sociocultural Norm Similarities and Differences via Situational Alignment and Explainable Textual Entailment

Sky CH-Wang, Arkadiy Saakyan, Oliver Li, Zhou Yu, Smaranda Muresan

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

Comments EMNLP 2023 Main Conference (Long Paper)

LLM-powered Data Augmentation for Enhanced Cross-lingual Performance

Chenxi Whitehouse, Monojit Choudhury, Alham Fikri Aji

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments EMNLP 2023 Main Conference

HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models

Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to EMNLP 2023 Main Conference (Long Paper)

MEGA: Multilingual Evaluation of Generative AI

Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Maxamed Axmed, Kalika Bali, Sunayana Sitaram

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments EMNLP 2023

Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization

Chenhui Shen, Liying Cheng, Xuan-Phi Nguyen, Yang You, Lidong Bing

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 19 pages, 5 figures

Journal ref Findings of EMNLP 2023

A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction

Ruihao Shui, Yixin Cao, Xiang Wang, Tat-Seng Chua

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments EMNLP Findings 2023

Visual Storytelling with Question-Answer Plans

Danyang Liu, Mirella Lapata, Frank Keller

专题命中 推理评测 ：planning(abstract)；分类 cs.CL

Comments EMNLP 2023 Findings

Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physics

Jason Holmes, Zhengliang Liu, Lian Zhang, Yuzhen Ding, Terence T. Sio, Lisa A. McGee, Jonathan B. Ashman, Xiang Li, Tianming Liu, Jiajian Shen, Wei Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Lion: Adversarial Distillation of Proprietary Large Language Models

Yuxin Jiang, Chunkit Chan, Mingyang Chen, Wei Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 21 pages, 5 figures, EMNLP 2023 main conference

MM-BigBench: Evaluating Multimodal Models on Multimodal Content Comprehension Tasks

Xiaocui Yang, Wenfang Wu, Shi Feng, Ming Wang, Daling Wang, Yang Li, Qi Sun, Yifei Zhang, Xiaoming Fu, Soujanya Poria

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Underview

Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research

Cole Gulino, Justin Fu, Wenjie Luo, George Tucker, Eli Bronstein, Yiren Lu, Jean Harb, Xinlei Pan, Yan Wang, Xiangyu Chen, John D. Co-Reyes, Rishabh Agarwal, Rebecca Roelofs, Yao Lu, Nico Montali, Paul Mougin, Zoey Yang, Brandyn White, Aleksandra Faust, Rowan McAllister, Dragomir Anguelov, Benjamin Sapp

专题命中 推理评测 ：planning(abstract)；分类 cs.LG

Being Right for Whose Right Reasons?

Terne Sasha Thorn Jakobsen, Laura Cabello, Anders Søgaard

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments In Proceedings of ACL 2023

Harnessing Large Language Models' Empathetic Response Generation Capabilities for Online Mental Health Counselling Support

Siyuan Brandon Loh, Aravind Sesagiri Raamkumar

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 7 pages, 1 figure

TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models

Xiao Wang, Yuansen Zhang, Tianze Chen, Songyang Gao, Senjie Jin, Xianjun Yang, Zhiheng Xi, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, Xuanjing Huang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Fine-grained Audio-Visual Joint Representations for Multimodal Large Language Models

Guangzhi Sun, Wenyi Yu, Changli Tang, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, Chao Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Text Classification via Large Language Models

Xiaofei Sun, Xiaoya Li, Jiwei Li, Fei Wu, Shangwei Guo, Tianwei Zhang, Guoyin Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Pre-print Version

Zero-Resource Hallucination Prevention for Large Language Models

Junyu Luo, Cao Xiao, Fenglong Ma

专题命中 推理评测 ：CoT(abstract)；分类 cs.CL

InstructProtein: Aligning Human and Protein Language via Knowledge Instruction

Zeyuan Wang, Qiang Zhang, Keyan Ding, Ming Qin, Xiang Zhuang, Xiaotong Li, Huajun Chen

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

L-Eval: Instituting Standardized Evaluation for Long Context Language Models

Chenxin An, Shansan Gong, Ming Zhong, Xingjian Zhao, Mukai Li, Jun Zhang, Lingpeng Kong, Xipeng Qiu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Building and Road Segmentation Using EffUNet and Transfer Learning Approach

Sahil Gangurde

专题命中 推理评测 ：planning(abstract)；分类 cs.LG

Comments The transformer network analysis was not included in the current paper