语言大模型 / LLM

Behavior Injection: Preparing Language Models for Reinforcement Learning

Zhepeng Cen, Yihang Yao, William Han, Zuxin Liu, Ding Zhao

机构 * Carnegie Mellon University（卡内基梅隆大学） ； Salesforce AI Research（Salesforce人工智能研究） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；post-training(abstract)；SFT(abstract)

Comments NeurIPS 2025

BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining

DatologyAI, :, Pratyush Maini, Vineeth Dorna, Parth Doshi, Aldo Carranza, Fan Pan, Jack Urbanek, Paul Burstein, Alex Fang, Alvin Deng, Amro Abbas, Brett Larsen, Cody Blakeney, Charvi Bannur, Christina Baek, Darren Teh, David Schwab, Haakon Mongstad, Haoli Yin, Josh Wills, Kaleigh Mentzer, Luke Merrick, Ricardo Monti, Rishabh Adiga, Siddharth Joshi, Spandan Das, Zhengping Wang, Bogdan Gaza, Ari Morcos, Matthew Leavitt

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Blog version can be viewed at: http://blog.datologyai.com/beyondweb

A Unified Speech LLM for Diarization and Speech Recognition in Multilingual Conversations

Phurich Saengthong, Boonnithi Jiaramaneepinit, Sheng Li, Manabu Okumura, Takahiro Shinozaki

机构 * Institute of Science Tokyo（东京科学研究所） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；SLM(abstract)

Can We Predict the Unpredictable? Leveraging DisasterNet-LLM for Multimodal Disaster Classification

Manaswi Kulahara, Gautam Siddharth Kashyap, Nipun Joshi, Arpita Soni

机构 * TERI School Of Advanced Studies（TERI高级研究学院） ； Macquarie University（麦考瑞大学） ； Cornell University（康奈尔大学） ； Eudoxia Research University（欧多西亚研究大学） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments Accepted in the 2025 IEEE International Geoscience and Remote Sensing Symposium (IGARSS 2025), scheduled for 3 - 8 August 2025 in Brisbane, Australia

Biomed-Enriched: A Biomedical Dataset Enriched with LLMs for Pretraining and Extracting Rare and Hidden Content

Rian Touchent, Nathan Godey, Eric de la Clergerie

机构 * Sorbonne Université（索邦大学） ； INRIA Paris（巴黎国家信息与自动化研究所） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；small language model(abstract)

Comments Dataset link: https://hf.co/datasets/almanach/Biomed-Enriched

Parsing the Switch: LLM-Based UD Annotation for Complex Code-Switched and Low-Resource Languages

Olga Kellert, Nemika Tyagi, Muhammad Imran, Nelvin Licona-Guevara, Carlos Gómez-Rodríguez

机构 * Arizona State University（亚利桑那州立大学） ； Universidade da Coruña, CITIC（科鲁纳大学，CITIC） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 16 pages

MoDoMoDo: Multi-Domain Data Mixtures for Multimodal LLM Reinforcement Learning

Yiqing Liang, Jielin Qiu, Wenhao Ding, Zuxin Liu, James Tompkin, Mengdi Xu, Mengzhou Xia, Zhengzhong Tu, Laixi Shi, Jiacheng Zhu

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

Comments Project Webpage: https://modomodo-rl.github.io/

GeoVision Labeler: Zero-Shot Geospatial Classification with Vision and Language Models

Gilles Quentin Hacheme, Girmaw Abebe Tadesse, Caleb Robinson, Akram Zaytar, Rahul Dodhia, Juan M. Lavista Ferres

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；pretraining(abstract)

Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game

Zelai Xu, Chao Yu, Fei Fang, Yu Wang, Yi Wu

机构 * Zelai Xu（泽来许） ； Chao Yu（姚朝） ； Fei Fang（方飞） ； Yu Wang（王宇） ； Yi Wu（吴毅） 纠错

专题命中 预训练与数据 ：language agent(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Published in ICML 2024

Distilling Closed-Source LLM's Knowledge for Locally Stable and Economic Biomedical Entity Linking

Yihao Ai, Zhiyuan Ning, Weiwei Dai, Pengfei Wang, Yi Du, Wenjuan Cui, Kunpeng Liu, Yuanchun Zhou

机构 * Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心） ； University of Chinese Academy of Sciences（中国科学院大学） ； Changsha Aier Eye Hospital（长沙爱尔眼科医院） ； Portland State University（波特兰州立大学） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Accepted by ICIC 2025

Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice

Jian-Qiao Zhu, Haijiang Yan, Thomas L. Griffiths

机构 * Department of Computer Science（计算机科学系） ； Princeton University（普林斯顿大学） ； Department of Psychology（心理学系） ； University of Warwick（沃里克大学） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；pretraining(abstract)

Journal ref ICLR 2025

United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Elections

Leah von der Heyde, Anna-Carolina Haensch, Alexander Wenz, Bolei Ma

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens

Jiacheng Liu, Sewon Min, Luke Zettlemoyer, Yejin Choi, Hannaneh Hajishirzi

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；pretraining(abstract)

Comments Published at COLM 2024, spotlight paper

Mixture-of-Personas Language Models for Population Simulation

Ngoc Bui, Hieu Trung Nguyen, Shantanu Kumar, Julian Theodore, Weikang Qiu, Viet Anh Nguyen, Rex Ying

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；prompting(abstract)

The Lucie-7B LLM and the Lucie Training Dataset: Open resources for multilingual language generation

Olivier Gouvert, Julie Hunter, Jérôme Louradour, Christophe Cerisara, Evan Dufraisse, Yaya Sy, Laura Rivière, Jean-Pierre Lorré, OpenLLM-France community

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；foundation model(abstract)

GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphs

Yun Zhu, Haizhou Shi, Xiaotang Wang, Yongchao Liu, Yaoke Wang, Boci Peng, Chuntao Hong, Siliang Tang

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments Accepted to WWW'25

Omni-DNA: A Unified Genomic Foundation Model for Cross-Modal and Multi-Task Learning

Zehui Li, Vallijah Subasri, Yifei Shen, Dongsheng Li, Yiren Zhao, Guy-Bart Stan, Caihua Shan

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervision

Yunyi Zhang, Ruozhen Yang, Xueqiang Xu, Rui Li, Jinfeng Xiao, Jiaming Shen, Jiawei Han

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Accepted to WWW 2025 Research Track

BTS: Harmonizing Specialized Experts into a Generalist LLM

Qizhen Zhang, Prajjwal Bhargava, Chloe Bi, Chris X. Cai, Jakob Foerster, Jeremy Fu, Punit Singh Koura, Ruan Silva, Sheng Shen, Emily Dinan, Suchin Gururangan, Mike Lewis

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Unlocking Tuning-Free Few-Shot Adaptability in Visual Foundation Models by Recycling Pre-Tuned LoRAs

Zixuan Hu, Yongxian Wei, Li Shen, Chun Yuan, Dacheng Tao

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Crystal: Illuminating LLM Abilities on Language and Code

Tianhua Tao, Junbo Li, Bowen Tan, Hongyi Wang, William Marshall, Bhargav M Kanakiya, Joel Hestness, Natalia Vassilieva, Zhiqiang Shen, Eric P. Xing, Zhengzhong Liu

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments Published as a conference paper at COLM 2024

Language Agents Meet Causality -- Bridging LLMs and Causal World Models

John Gkountouras, Matthias Lindemann, Phillip Lippe, Efstratios Gavves, Ivan Titov

专题命中 预训练与数据 ：language agent(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Project page: https://j0hngou.github.io/LLMCWM/

OpenGraph: Towards Open Graph Foundation Models

Lianghao Xia, Ben Kao, Chao Huang

专题命中 预训练与数据 ：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by EMNLP'2024

A multimodal LLM for the non-invasive decoding of spoken text from brain recordings

Youssef Hmamouche, Ismail Chihab, Lahoucine Kdouri, Amal El Fallah Seghrouchni

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments 15 pages, 4 figures

dnaGrinder: a lightweight and high-capacity genomic foundation model

Qihang Zhao, Chi Zhang, Weixiong Zhang

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Source-Aware Training Enables Knowledge Attribution in Language Models

Muhammad Khalifa, David Wadden, Emma Strubell, Honglak Lee, Lu Wang, Iz Beltagy, Hao Peng

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；pretraining(abstract)

Comments COLM '24

IGOT: Information Gain Optimized Tokenizer on Domain Adaptive Pretraining

Dawei Feng, Yihai Zhang, Zhixuan Xu

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web

Yibo Yan, Haomin Wen, Siru Zhong, Wei Chen, Haodong Chen, Qingsong Wen, Roger Zimmermann, Yuxuan Liang

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by The Web Conference 2024

Learning to Deliver: a Foundation Model for the Montreal Capacitated Vehicle Routing Problem

Samuel J. K. Chin, Matthias Winkenbach, Akash Srivastava

专题命中 预训练与数据 ：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method

Biao Zhang, Zhongtao Liu, Colin Cherry, Orhan Firat

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments ICLR24