语言大模型 / LLM - arXivDaily 专题

2601.12179 2026-01-21 cs.CL 86%

Tolerance Principle and Small Language Model Learning

容忍原则与小型语言模型学习

Adam E. Friedman, Stevan Harnad, Rushen Shi

专题命中预训练与数据：language model(title,abstract)；small language model(title)；分类 cs.CL

AI总结本研究通过训练小型语言模型BabyBERTa，探讨了规则泛化所需的最小训练数据量，发现其学习动态与杨提出的容忍原则不符。

Comments 14 pages, 6 figures. BUCLD 50 Proceedings. To be published in 2026 by Cascadilla Press

2310.19531 2026-01-16 cs.CL 86%

MiLe Loss: a New Entropy-Weighed Loss for Mitigating the Bias of Learning Difficulties in Large Language Models

MiLe Loss：一种新的熵加权损失，用于减轻大语言模型在学习困难方面的偏差

Zhenpeng Su, Xing Wu, Xue Bai, Zijia Lin, Hui Chen, Guiguang Ding, Wei Zhou, Songlin Hu

机构 * Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）； School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）； Kuaishou Technology（快手科技）； Tsinghua University（清华大学）

专题命中预训练与数据：language model(title,abstract)；large language model(title)；分类 cs.CL

AI总结 MiLe Loss通过熵加权机制减轻大语言模型在学习困难方面的偏差，提升模型对难学标记的关注度，从而在下游任务中取得更好的表现。

Comments This paper has been accepted by NAACL 2024

2404.08816 2026-01-14 cs.CL econ.EM 86%

Measuring the Quality of Answers in Political Q&As with Large Language Models

利用大语言模型评估政治问答中的答案质量

R. Michael Alvarez, Jacob Morrier

机构 * Division of the Humanities and Social Sciences（人文与社会科学系）； California Institute of Technology（加州理工学院）

专题命中预训练与数据：language model(title,abstract)；large language model(title)；分类 cs.CL

AI总结本文提出利用大语言模型评估政治问答答案质量的方法，通过语义相关性衡量答案的相关性和深度，并发现答案质量与提问议员政党存在相关性。

Journal ref Polit. Anal. 34 (2026) 78-95

2411.16380 2025-12-18 eess.IV cs.AI cs.CV 86%

From Pretraining to Privacy: Federated Ultrasound Foundation Model with Self-Supervised Learning

从预训练到隐私：具有自监督学习的联邦超声基础模型

Yuncheng Jiang, Chun-Mei Feng, Jinke Ren, Jun Wei, Zixun Zhang, Yiwen Hu, Yunbi Liu, Rui Sun, Xuemei Tang, Juan Du, Xiang Wan, Yong Xu, Bo Du, Xin Gao, Guangyu Wang, Shaohua Zhou, Shuguang Cui, Zhen Li

机构 * FNii-Shenzhen, The Chinese University of Hong Kong, Shenzhen（深圳FNii，香港中文大学（深圳））； School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（科学与工程学院，香港中文大学（深圳））； West China Hospital of Sichuan University, Chengdu（四川大学华西医院）； School of Computer Science, University College Dublin, Ireland（计算机科学学院，都柏林大学（爱尔兰））； College of Computer Science and Software Engineering, Shenzhen University, Shenzhen（计算机科学与软件工程学院，深圳大学）； South China Hospital, Health Science Center, Shenzhen University, Shenzhen（南方医院，深圳大学健康科学中心）； School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing（计算机科学学院，南京邮电大学）； Affiliated Hospital of North Sichuan Medical College, Sichuan（北川医学院附属医院）； North Sichuan Medical College, Sichuan（北川医学院）； Shenzhen Research Institute of Big Data, Shenzhen（深圳大数据研究院）； Bio-Computing Research Center, Harbin Institute of Technology, Shenzhen（生物计算研究中心，哈尔滨工业大学（深圳））； School of Computer Science, Wuhan University, Wuhan（计算机科学学院，武汉大学）； Computer Science Program, Computer, E（计算机科学项目，计算机）

专题命中预训练与数据：foundation model(title,abstract)；pretraining(title)；分类 cs.AI

AI总结 UltraFedFM通过联邦学习和自监督学习，构建了一个隐私保护的超声基础模型，实现了在多种疾病诊断和病变分割上的高准确率，超越了中级医师水平，达到了专家级表现。

Comments npj digital medicine(2025)

2509.17289 2025-11-13 cs.CL 86%

Automated Knowledge Graph Construction using Large Language Models and Sentence Complexity Modelling

Sydney Anuyah, Mehedi Mahmud Kaushik, Krishna Dwarampudi, Rakesh Shiradkar, Arjan Durresi, Sunandan Chakraborty

机构 * Indiana University（印第安纳大学）； Purdue University（普渡大学）； Department of Biomedical Engineering（生物医学工程系）； Informatics, Indiana University（印第安纳大学信息学院）

专题命中预训练与数据：large language model(title)；language model(title)；prompting(abstract)；分类 cs.CL

2511.03570 2025-11-06 cs.LG 86%

TabGemma: Text-Based Tabular ICL via LLM using Continued Pretraining and Retrieval

Günther Schindler, Maximilian Schambach, Michael Medek, Sam Thelin

机构 * SAP SE（SAP公司）

专题命中预训练与数据：pretraining(title,abstract)；LLM(title)；分类 cs.LG

2510.25273 2025-10-30 cs.CL 86%

Adapting Small Language Models to Low-Resource Domains: A Case Study in Hindi Tourism QA

Sandipan Majhi, Paheli Bhattacharya

机构 * Indian Institute of Technology Kharagpur, India（印度理工学院Kharagpur分校）； Bosch Research（博世研究）； Technology Centre, Bangalore, India（技术中心，班加罗尔，印度）

专题命中预训练与数据：language model(title,abstract)；small language model(title)；分类 cs.CL

Comments Accepted at the Forum for Information Retrieval Evaluation 2025 (VATIKA Track)

2510.20475 2025-10-24 cs.CL 86%

Mask and You Shall Receive: Optimizing Masked Language Modeling For Pretraining BabyLMs

Lukas Edman, Alexander Fraser

机构 * School of Computation, Information and Technology, TU Munich（计算、信息与技术学院，慕尼黑工业大学）； Munich Center for Machine Learning（慕尼黑机器学习中心）； Munich Data Science Institute（慕尼黑数据科学研究所）

专题命中预训练与数据：language model(title,abstract)；pretraining(title)；分类 cs.CL

Comments Submission to the 2025 BabyLM Challenge

2510.09423 2025-10-13 cs.LG 86%

Weight Initialization and Variance Dynamics in Deep Neural Networks and Large Language Models

Yankun Han

机构 * University of Florida（佛罗里达大学）

专题命中预训练与数据：large language model(title)；language model(title)；pretraining(abstract)；分类 cs.LG

Comments 8 pages, 6 figures

2509.25283 2025-10-01 cs.CY cs.AI cs.HC 86%

Effectiveness of Large Language Models in Simulating Regional Psychological Structures: An Empirical Examination of Personality and Subjective Well-being

Ke Luoma, Li Zengyi, Liao Jiangqun, Tong Song, Peng Kaiping

专题命中预训练与数据：large language model(title)；language model(title)；LLM(abstract)；分类 cs.AI

2403.13638 2025-07-08 cs.CL 86%

Pretraining Language Models Using Translationese

Meet Doshi, Raj Dabre, Pushpak Bhattacharyya

机构 * cse.iitb.ac.in（印度理工学院班加罗尔分校计算机科学与工程系）； nict.go.jp（日本国立信息与通信技术研究所）

专题命中预训练与数据：language model(title,abstract)；pretraining(title)；分类 cs.CL

2507.02873 2025-07-08 math.HO cs.AI 86%

Using Large Language Models to Study Mathematical Practice

William D'Alessandro

机构 * William D’Alessandro（独立研究者）

专题命中预训练与数据：large language model(title)；language model(title)；LLM(abstract)；分类 cs.AI

2412.06239 2024-12-10 cs.CR cs.AI 86%

Unseen Attack Detection in Software-Defined Networking Using a BERT-Based Large Language Model

Mohammed N. Swileh, Shengli Zhang

专题命中预训练与数据：language model(title,abstract)；large language model(title)；分类 cs.AI

Comments Mohammed N. Swileh is first author. Shengli Zhang is corresponding author

2404.17808 2024-11-14 cs.CL 86%

Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token Removal

Haoran Lian, Yizhe Xiong, Jianwei Niu, Shasha Mo, Zhenpeng Su, Zijia Lin, Hui Chen, Peng Liu, Jungong Han, Guiguang Ding

专题命中预训练与数据：language model(title,abstract)；large language model(title)；分类 cs.CL

2410.19775 2024-10-29 cs.CY cs.AI 86%

Gender Bias of LLM in Economics: An Existentialism Perspective

Hui Zhong, Songsheng Chen, Mian Liang

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract,comments)；language model(abstract,comments)；分类 cs.AI

Comments Gender Bias, Large Language Models, Decision-Making

2410.16121 2024-10-22 cs.LG cs.CR 86%

Extracting Spatiotemporal Data from Gradients with Large Language Models

Lele Zheng, Yang Cao, Renhe Jiang, Kenjiro Taura, Yulong Shen, Sheng Li, Masatoshi Yoshikawa

专题命中预训练与数据：language model(title,abstract)；large language model(title)；分类 cs.LG

Comments arXiv admin note: substantial text overlap with arXiv:2407.08529

2405.03821 2024-05-08 cs.HC cs.AI cs.SE 86%

Thoughtful Things: Building Human-Centric Smart Devices with Small Language Models

Evan King, Haoxiang Yu, Sahil Vartak, Jenna Jacob, Sangsu Lee, Christine Julien

专题命中预训练与数据：language model(title,abstract)；small language model(title)；分类 cs.AI

Comments 24 pages (3 pages of references)

2403.07118 2024-04-09 cs.CL 86%

Narrating Causal Graphs with Large Language Models

Atharva Phatak, Vijay K. Mago, Ameeta Agrawal, Aravind Inbasekaran, Philippe J. Giabbanelli

专题命中预训练与数据：language model(title,abstract)；large language model(title)；分类 cs.CL

Comments HICSS '24

Journal ref Proceedings of the 57th Hawaii International Conference on System Sciences 2024

2403.19783 2024-04-01 cond-mat.mtrl-sci cs.LG 86%

AlloyBERT: Alloy Property Prediction with Large Language Models

Akshat Chaudhari, Chakradhar Guntuboina, Hongshuo Huang, Amir Barati Farimani

专题命中预训练与数据：language model(title,abstract)；large language model(title)；分类 cs.LG

Comments 20 pages, 3 figures

2309.08765 2023-12-20 q-bio.QM cs.LG 86%

Mining Patents with Large Language Models Elucidates the Chemical Function Landscape

Clayton W. Kosonocky, Claus O. Wilke, Edward M. Marcotte, Andrew D. Ellington

专题命中预训练与数据：large language model(title)；language model(title)；LLM(abstract)；分类 cs.LG

Comments Under review

2304.03738 2023-11-14 cs.CY cs.CL 86%

Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models

Emilio Ferrara

专题命中预训练与数据：language model(title,abstract)；large language model(title)；分类 cs.CL

Comments Published on First Monday https://firstmonday.org/ojs/index.php/fm/article/view/13346/11365

Journal ref First Monday, Volume 28, Number 11 - 6 November 2023

2304.13994 2023-06-23 cs.CL 86%

SweCTRL-Mini: a data-transparent Transformer-based large language model for controllable text generation in Swedish

Dmytro Kalpakchi, Johan Boye

专题命中预训练与数据：language model(title,abstract)；large language model(title)；分类 cs.CL

Comments Added information about training tokenizer

2305.13698 2023-05-24 cs.CL 86%

Exploring Large Language Models for Classical Philology

Frederick Riemenschneider, Anette Frank

专题命中预训练与数据：language model(title,abstract)；large language model(title)；分类 cs.CL

Comments Paper accepted for publication at ACL 2023 Main; 10 pages, 7 appendix pages, 4 figures, 13 tables

2205.11758 2022-10-25 cs.CL 86%

Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models

Terra Blevins, Hila Gonen, Luke Zettlemoyer

专题命中预训练与数据：pretraining(title,abstract)；language model(title)；分类 cs.CL

Comments EMNLP 2022

2209.12711 2022-09-27 cs.CL 86%

Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts

Joel Jang, Seonghyeon Ye, Minjoon Seo

专题命中预训练与数据：language model(title,abstract)；large language model(title)；分类 cs.CL

2101.00204 2022-05-11 cs.CL 86%

BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla

Abhik Bhattacharjee, Tahmid Hasan, Wasi Uddin Ahmad, Kazi Samin, Md Saiful Islam, Anindya Iqbal, M. Sohel Rahman, Rifat Shahriyar

专题命中预训练与数据：pretraining(title,abstract)；language model(title)；分类 cs.CL

Comments Findings of North American Chapter of the Association for Computational Linguistics, NAACL 2022 (camera-ready)

2202.05144 2022-02-11 cs.CL 86%

InPars: Data Augmentation for Information Retrieval using Large Language Models

Luiz Bonifacio, Hugo Abonizio, Marzieh Fadaee, Rodrigo Nogueira

专题命中预训练与数据：language model(title,abstract)；large language model(title)；分类 cs.CL

2012.15283 2021-09-20 cs.CL 86%

ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning

Rujun Han, Xiang Ren, Nanyun Peng

专题命中预训练与数据：language model(title,abstract)；pretraining(title)；分类 cs.CL

Comments This paper has been accepted by EMNLP'21 main conference

2608.13517 2026-08-24 cs.CL cs.AI 版本更新 86%

DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training Data

DFM Mimir v1：仅使用许可的后训练数据，在10亿参数规模下实现前沿性能的开放HRM模型

Peter Schneider-Kamp, Jacob Nielsen, Gianluca Barmina, Kenneth Enevoldsen, Lukas Galke Poech

专题命中预训练与数据：post-training(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结研究人员因大模型开发依赖非许可数据受阻，本文提出仅用许可后训练数据训练的10亿参数HRM模型Mimir v1，其在多基准测试中性能优于同规模HRM-Text 1B，可与更大前沿模型媲美，且在丹麦语任务达新SOTA，已发布于Hugging Face Hub

Comments Technical Report, 20 Pages, 1 Model, Hierarchical Reasoning Model, v2: added memorization audits

2607.16669 2026-07-21 cs.CL cs.LG cs.SE 新提交 86%

OpenLanguageModel: Readable and Composable Small-Language-Model Pretraining for Education and Research

开放语言模型：用于教育和研究的可读且可组合的小语言模型预训练

Tavish Mankash, Vardhaman Kalloli, Keshava Prasad, Deepan Muthirayan

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；small language model(abstract)；分类 cs.CL、cs.LG

AI总结 OpenLanguageModel是开源PyTorch库，用于构建和预训练小语言模型。其模型代码易读，能连接多种组件。通过追踪GPT - 2展示完整路径，含27个预设和文档。验证有高一致性和效率等成果，采用MIT许可，可多途径获取。

Comments 8 pages, 3 figures, and 1 table. Website: https://openlanguagemodel.github.io/openlanguagemodel/. Code: https://github.com/openlanguagemodel/openlanguagemodel

1. 预训练与数据 12429 篇

Tolerance Principle and Small Language Model Learning

MiLe Loss: a New Entropy-Weighed Loss for Mitigating the Bias of Learning Difficulties in Large Language Models

Measuring the Quality of Answers in Political Q&As with Large Language Models

From Pretraining to Privacy: Federated Ultrasound Foundation Model with Self-Supervised Learning

Automated Knowledge Graph Construction using Large Language Models and Sentence Complexity Modelling

TabGemma: Text-Based Tabular ICL via LLM using Continued Pretraining and Retrieval

Adapting Small Language Models to Low-Resource Domains: A Case Study in Hindi Tourism QA

Mask and You Shall Receive: Optimizing Masked Language Modeling For Pretraining BabyLMs

Weight Initialization and Variance Dynamics in Deep Neural Networks and Large Language Models

Effectiveness of Large Language Models in Simulating Regional Psychological Structures: An Empirical Examination of Personality and Subjective Well-being

Pretraining Language Models Using Translationese

Using Large Language Models to Study Mathematical Practice

Unseen Attack Detection in Software-Defined Networking Using a BERT-Based Large Language Model

Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token Removal

Gender Bias of LLM in Economics: An Existentialism Perspective

Extracting Spatiotemporal Data from Gradients with Large Language Models

Thoughtful Things: Building Human-Centric Smart Devices with Small Language Models

Narrating Causal Graphs with Large Language Models

AlloyBERT: Alloy Property Prediction with Large Language Models

Mining Patents with Large Language Models Elucidates the Chemical Function Landscape

Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models

SweCTRL-Mini: a data-transparent Transformer-based large language model for controllable text generation in Swedish

Exploring Large Language Models for Classical Philology

Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models

Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts

BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla

InPars: Data Augmentation for Information Retrieval using Large Language Models

ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning

DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training Data

OpenLanguageModel: Readable and Composable Small-Language-Model Pretraining for Education and Research