Recent studies have demonstrated outstanding capabilities of large language models (LLMs) in software engineering tasks, including code generation and comprehension. While LLMs have shown significant potential in assisting with coding, LLMs are vulnerable to adversarial attacks. In this paper, we investigate the vulnerability of LLMs to imperceptible attacks. This class of attacks manipulate source code at the character level, which renders the changes invisible to human reviewers yet effective in misleading LLMs' behaviour. We devise these attacks into four distinct categories and analyse their impacts on code analysis and comprehension tasks. These four types of imperceptible character attacks include coding reordering, invisible coding characters, code deletions, and code homoglyphs. To assess the robustness of state-of-the-art LLMs, we present a systematic evaluation across multiple models using both perturbed and clean code snippets. Two evaluation metrics, model confidence using log probabilities of response and response correctness, are introduced. The results reveal that LLMs are susceptible to imperceptible coding perturbations, with varying degrees of degradation highlighted across different LLMs. Furthermore, we observe a consistent negative correlation between perturbation magnitude and model performance. These results highlight the urgent need for robust LLMs capable of manoeuvring behaviours under imperceptible adversarial conditions.
Can LLMs Augment Low-Resource Reading Comprehension Datasets? Opportunities and Challenges
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Carnegie Mellon University（卡内基梅隆大学）；Georgia Institute of Technology（佐治亚理工学院）；Stanford University（斯坦福大学）；Amazon AI（亚马逊人工智能）
ASMA-Tune: Unlocking LLMs' Assembly Code Comprehension via Structural-Semantic Instruction Tuning
Nankai University（南开大学）；Zhejiang University（浙江大学）；Ant Group（蚂蚁集团）；Renmin University of China（中国人民大学）；University of the Chinese Academy of Sciences（中国科学院大学）
Comparing Human and Language Models Sentence Processing Difficulties on Complex Structures
Blavatnik School of Computer Science, Tel Aviv University, Israel（巴尔-艾塔夫大学计算机科学学院）；Department of Linguistics, Tel Aviv University, Israel（巴尔-艾塔夫大学语言学系）；Sagol School of Neuroscience, Tel Aviv University, Israel（巴尔-艾塔夫大学神经科学学院）
Independent Mobility GPT (IDM-GPT): A Self-Supervised Multi-Agent Large Language Model Framework for Customized Traffic Mobility Analysis Using Machine Learning Models
University of Utah（犹他大学）；University of Washington（华盛顿大学）
针对交通大数据分析成本高及隐私问题，研究提出基于大语言模型的多智能体框架 IDM-GPT，通过连接用户、数据库与 ML 模型实现定制化交通分析与管理建议，使非专业用户能近实时获取分析结果，在多项交通任务中表现优异。
Integrating LLM, EEG, and Eye-Tracking Biomarker Analysis for Word-Level Neural State Classification in Semantic Inference Reading Comprehension
Brown University（布朗大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）；Yale University（耶鲁大学）；University of California Los Angeles（加利福尼亚大学洛杉矶分校）；National Central University（国立中央大学）
RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension
Xinjiang University（新疆大学）；Renmin University of China（中国人民大学）；Anhui University（安徽大学）；Sun Yat-sen University（中山大学）；Tsinghua University（清华大学）；University of Southampton（南安普顿大学）
Predicting Learning Performance with Large Language Models: A Study in Adult Literacy
Institute for Intelligent Systems, University of Memphis（孟菲斯大学智能系统研究所）；University of Memphis（孟菲斯大学）；Human-Computer Interaction Institute, Carnegie Mellon University（卡内基梅隆大学人机交互研究所）；Carnegie Mellon University（卡内基梅隆大学）；Hong Kong Polytechnic University（香港理工大学）
We propose a generative machine comprehension model that learns jointly to ask and answer questions based on documents. The proposed model uses a sequence-to-sequence framework that encodes the document and generates a question (answer) given an answer (question). Significant improvement in model performance is observed empirically on the SQuAD corpus, confirming our hypothesis that the model benefits from jointly learning to perform both tasks. We believe the joint model's novelty offers a new perspective on machine comprehension beyond architectural engineering, and serves as a first step to…
School of Computer Science（计算机学院）；Fudan University（复旦大学）；Academy for Engineering Technology（工程技术研究院）；Shanghai Children’s Hospital of Fudan University（复旦大学附属儿科医院）；National Children’s Medical Center（国家儿童医学中心）
LProtector: An LLM-driven Vulnerability Detection System
University of Pennsylvania（宾夕法尼亚大学）；University of Electronic Science & Technology of China（电子科技大学）；Texas A&M University（德克萨斯农工大学）；Georgetown University（乔治城大学）；Northern Arizona University（北亚利桑那大学）；LG Energy Solution (Nanjing) Co., Ltd（LG新能源（南京）有限公司）
University of Washington（华盛顿大学）；Westlake University（西湖大学）；Zhejiang University（浙江大学）
Contextualized representations trained over large raw text data have given remarkable improvements for NLP tasks including question answering and reading comprehension. There have been works showing that syntactic, semantic and word sense knowledge are contained in such representations, which explains why they benefit such tasks. However, relatively little work has been done investigating commonsense knowledge contained in contextualized representations, which is crucial for human question answering and reading comprehension. We study the commonsense ability of GPT, BERT, XLNet, and RoBERTa by…
