语言大模型 / LLM

Reverse That Number! Decoding Order Matters in Arithmetic Learning

Daniel Zhang-Li, Nianyi Lin, Jifan Yu, Zheyuan Zhang, Zijun Yao, Xiaokang Zhang, Lei Hou, Jing Zhang, Juanzi Li

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)

Shenglai Zeng, Jiankun Zhang, Pengfei He, Yue Xing, Yiding Liu, Han Xu, Jie Ren, Shuaiqiang Wang, Dawei Yin, Yi Chang, Jiliang Tang

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations

Yudong Xu, Wenhao Li, Pashootan Vaezipoor, Scott Sanner, Elias B. Khalil

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 26 pages, 15 figures, published in Transactions on Machine Learning Research (TMLR)

Multi-Step Dialogue Workflow Action Prediction

Ramya Ramakrishnan, Ethan R. Elenberg, Hashan Narangodage, Ryan McDonald

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

NESTLE: a No-Code Tool for Statistical Analysis of Legal Corpus

Kyoungyeon Cho, Seungkum Han, Young Rok Choi, Wonseok Hwang

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments EACL 2024 System Demonstration Track

POMP: Probability-driven Meta-graph Prompter for LLMs in Low-resource Unsupervised Neural Machine Translation

Shilong Pan, Zhiliang Tian, Liang Ding, Zhen Huang, Zhihua Wen, Dongsheng Li

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Pre-training LLMs using human-like development data corpus

Khushi Bhardwaj, Raj Sanjay Shah, Sashank Varma

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning

Jatmo: Prompt Injection Defense by Task-Specific Finetuning

Julien Piet, Maha Alrashed, Chawin Sitawarin, Sizhe Chen, Zeming Wei, Elizabeth Sun, Basel Alomair, David Wagner

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；instruction tuning(abstract)；分类 cs.CL、cs.AI

Comments 24 pages, 6 figures

Beyond English: Evaluating LLMs for Arabic Grammatical Error Correction

Sang Yun Kwon, Gagan Bhatia, El Moatez Billah Nagoudi, Muhammad Abdul-Mageed

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments arXiv admin note: text overlap with arXiv:2308.04492

Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia

Alexander Sasha Vezhnevets, John P. Agapiou, Avia Aharon, Ron Ziv, Jayd Matyas, Edgar A. Duéñez-Guzmán, William A. Cunningham, Simon Osindero, Danny Karmon, Joel Z. Leibo

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 32 pages, 5 figures

Nonparametric Variational Regularisation of Pretrained Transformers

Fabio Fehr, James Henderson

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；post-training(abstract)；分类 cs.CL、cs.LG

Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMs

Xiang Zhang, Senyu Li, Bradley Hauer, Ning Shi, Grzegorz Kondrak

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments Paper accepted to EMNLP 2023

Who's Harry Potter? Approximate Unlearning in LLMs

Ronen Eldan, Mark Russinovich

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

MemoNet: Memorizing All Cross Features' Representations Efficiently via Multi-Hash Codebook Network for CTR Prediction

Pengtao Zhang, Junlin Zhang

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Journal ref ACM International Conference on Information and Knowledge Management(CIKM '23), October 21-25,2023,Birmingham,United Kingdom

Biomedical Entity Linking with Triple-aware Pre-Training

Xi Yan, Cedric Möller, Ricardo Usbeck

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

PRODIGY: Enabling In-context Learning Over Graphs

Qian Huang, Hongyu Ren, Peng Chen, Gregor Kržmanc, Daniel Zeng, Percy Liang, Jure Leskovec

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval

Yue Yu, Yuchen Zhuang, Rongzhi Zhang, Yu Meng, Jiaming Shen, Chao Zhang

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments ACL 2023 Findings (Code: https://github.com/yueyu1030/ReGen)

Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runs

Kelvin Guu, Albert Webson, Ellie Pavlick, Lucas Dixon, Ian Tenney, Tolga Bolukbasi

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Information

Yen-Ting Lin, Alexandros Papangelis, Seokhwan Kim, Sungjin Lee, Devamanyu Hazarika, Mahdi Namazifar, Di Jin, Yang Liu, Dilek Hakkani-Tur

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments Accepted at EACL 2023

Language Quantized AutoEncoders: Towards Unsupervised Text-Image Alignment

Hao Liu, Wilson Yan, Pieter Abbeel

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments Fixed typos

An Explanation of In-context Learning as Implicit Bayesian Inference

Sang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu Ma

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments ICLR 2022

Multitask Prompted Training Enables Zero-Shot Task Generalization

Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Tali Bers, Stella Biderman, Leo Gao, Thomas Wolf, Alexander M. Rush

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments ICLR 2022 Spotlight (with extended discussion)

Agent WARPP: Workflow Adherence via Runtime Parallel Personalization

Maria Emilia Mazzolenis, Ruirui Zhang

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted at the ICML 2025 Workshop on Multi-Agent Systems in the Era of Foundation Models: Opportunities, Challenges, and Futures. Code repo: https://github.com/emiliamazzo/WARPP/

Bridging Speech and Text: Enhancing ASR with Pinyin-to-Character Pre-training in LLMs

Yang Yuhang, Peng Yizhou, Eng Siong Chng, Xionghu Zhong

专题命中 预训练与数据 ：LLM(abstract,comments)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted by ISCSLP2024-Special session-Speech Processing in LLM Era

N-gram Injection into Transformers for Dynamic Language Model Adaptation in Handwritten Text Recognition

基于N-gram注入的Transformer在手写文本识别中的动态语言模型适应

Florent Meyer, Laurent Guichard, Yann Soullard, Denis Coquenet, Guillaume Gravier, Bertrand Coüasnon

机构 * ANTAI, Rennes, France（ANTAI研究院，法国里昂） ； Univ Rennes, CNRS, IRISA - UMR 6074, Rennes, France（里昂大学，法国国家科学研究中心，IRISA-UMR 6074，法国里昂） 纠错

专题命中 预训练与数据 ：language model(title,abstract)

AI总结 本文提出N-gram注入方法，用于在手写文本识别中动态调整Transformer的语言模型，以应对目标语料分布偏移问题，提升识别精度。

Comments Fix order of authors

Task-Adaptive 3D Cross-Field MRI Translation via Field-Conditioned Content-Style Pretraining

基于场条件内容-风格预训练的任务自适应3D跨场MRI转换

Haowen Pang, Yingqi Hao, Pengli Zhu

机构 * School of Integrated Circuits and Electronics, Beijing Institute of Technology（北京理工大学集成电路与电子学院） ； School of Biomedical Engineering, Tsinghua Medicine, Tsinghua University（清华大学医学院生物医学工程学院） ； Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)

AI总结 针对跨场MRI转换的域偏移问题，提出基于场条件内容-风格预训练的3D非配对转换框架，适配三项挑战赛任务，可保留三维解剖结构。

Comments MICCAI 2026 Workshop on MRIxFields

BAP-MOS: Bandit-Based Adaptive Prompting for Boundary-Sensitive Multi-Organ Segmentation

BAP-MOS：面向边界敏感型多器官分割的基于多臂老虎机的自适应提示

Satvik Praveen, Shengji Jin, Ahmed Lamidi, Xin Qian, Yi Sheng

专题命中 预训练与数据 ：prompting(title,abstract)

AI总结 该研究提出BAP-MOS自适应提示框架，将提示选择转化为多臂老虎机问题，在多器官超声分割基准上显著降低边界误差，泛化能力良好且无需修改模型主干。

Comments 9 pages, 5 figures. Source code available at https://github.com/SatvikPraveen/BAP-MOS

AgriField-40K: Adapting Vision Models to Agriculture With Efficient Continual Pretraining

AgriField-40K：通过高效持续预训练使视觉模型适配农业场景

Vasileios Tzouras, Paraskevas Pegios, Lazaros Nalpantidis

机构 * Technical University of Denmark (DTU)（丹麦技术大学（DTU）） ； Pioneer Centre for Artificial Intelligence（人工智能先锋中心） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)

AI总结 针对农业计算机视觉依赖昂贵标注与模型适配成本高的问题，构建含17种公开资源的AgriField-40K数据集，提出参数高效的AgriMAE方法，可少用9倍可训练参数实现与全微调相当甚至更优的下游性能，为农业视觉持续预训练提供实用资源。

Comments Accepted at the Computer Vision in Plant Phenotyping and Agriculture (CVPPA) Workshop at the European Conference on Computer Vision (ECCV) 2026

FlowCLIP: Contrastive Pretraining Using Domain Names for Encrypted Traffic Classification

FlowCLIP: 使用域名的对比预训练进行加密流量分类

Eun Hun Choi

专题命中 预训练与数据 ：pretraining(title,abstract)

AI总结 提出FlowCLIP框架，利用数据包侧信道特征（间隔、大小、方向）和CLIP对比目标对齐流量与域名表示，在QUIC流量数据集上跨周评估，优于基线方法。

Smartwatch Photoplethysmography-Derived Heart Age via ECG-Guided Cross-Modal Pretraining as a Digital Biomarker of Vascular Aging

通过心电引导跨模态预训练从智能手表光体积描记术衍生的心龄作为血管衰老的数字生物标志物

Donglin Xie, Xueying Gui, Yutian Zhu, Feng Xu, Guangkun Nie, Chenyang Xu, Jun Li, Shuailong Tang, Xiaoyu Li, Qi Xie, Yelei Li, Shenda Hong

专题命中 预训练与数据 ：pretraining(title,abstract)

AI总结 本研究开发心电引导跨模态框架，利用智能手表PPG衍生的心龄差作为数字生物标志物，证实其与动脉僵硬度、高血压显著相关，可用于血管衰老评估。