语言大模型 / LLM

Soundwave: Less is More for Speech-Text Alignment in LLMs

Yuhao Zhang, Zhiheng Liu, Fan Bu, Ruiyu Zhang, Benyou Wang, Haizhou Li

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

DSAI: Unbiased and Interpretable Latent Feature Extraction for Data-Centric AI

Hyowon Cho, Soonwon Ka, Daechul Park, Jaewook Kang, Minjoon Seo, Bokyung Son

机构 * KAIST AI（韩国科学技术院人工智能学院） ； NAVER AI Platform（NAVER人工智能平台） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Aligning Sentence Simplification with ESL Learner's Proficiency for Language Acquisition

Guanlin Li, Yuki Arase, Noel Crespi

机构 * Telecom SudParis, Institut Polytechnique de Paris（巴黎综合理工大学电信高等学院） ； School of Computing, Institute of Science Tokyo（东京科学大学计算机学院） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments NAACL2025 main

ControllableGPT: A Ground-Up Designed Controllable GPT for Molecule Optimization

Xuefeng Liu, Songhao Jiang, Bo Li, Rick Stevens

机构 * University of Chicago（芝加哥大学） ； Argonne National Laboratory（阿贡国家实验室） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Artificial Intelligence in Spectroscopy: Advancing Chemistry from Prediction to Generation and Beyond

Kehan Guo, Yili Shen, Gisela Abigail Gonzalez-Montiel, Yue Huang, Yujun Zhou, Mihir Surve, Zhichun Guo, Prayel Das, Nitesh V Chawla, Olaf Wiest, Xiangliang Zhang

机构 * University of Notre Dame（圣母大学） ； University of Washington（华盛顿大学） ； IBM Thomas J Watson Research Center（IBM托马斯·J·沃森研究中心） 纠错

专题命中 预训练与数据 ：foundation model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Matina: A Large-Scale 73B Token Persian Text Corpus

Sara Bourbour Hosseinbeigi, Fatemeh Taherinezhad, Heshaam Faili, Hamed Baghbani, Fatemeh Nadi, Mostafa Amiri

机构 * Tarbiat Modares University（塔比亚特·莫达雷斯大学） ； University of Tehran（德黑兰大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data

Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, Zhicheng Dou

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院） ； Microsoft Corporation（微软公司） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examples

Ajay Patel, Jiacheng Zhu, Justin Qiu, Zachary Horvitz, Marianna Apidianaki, Kathleen McKeown, Chris Callison-Burch

机构 * University of Pennsylvania（宾夕法尼亚大学） ； Columbia University（哥伦比亚大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments To appear at NAACL 2025

Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesis

Lin Yuan, Jun Xu, Honghao Gui, Mengshu Sun, Zhiqiang Zhang, Lei Liang, Jun Zhou

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by AAAI 2025

Transformers represent belief state geometry in their residual stream

Adam S. Shai, Sarah E. Marzen, Lucas Teixeira, Alexander Gietelink Oldenziel, Paul M. Riechers

机构 * Simplex（辛普莱克斯） ； PIBBSS ； Pitzer and Scripps College（皮策与斯克里普斯学院） ； University College London（伦敦大学学院） ； BITS Principles of Intelligent Behaviour in Biological and Social Systems（BITS生物与社会系统智能行为原理中心） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Fanar: An Arabic-Centric Multimodal Generative AI Platform

Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, Majd Hawasly, Mus'ab Husaini, Soon-Gyo Jung, Ji Kim Lucas, Walid Magdy, Safa Messaoud, Abubakr Mohamed, Tasnim Mohiuddin, Basel Mousi, Hamdy Mubarak, Ahmad Musleh, Zan Naeem, Mourad Ouzzani, Dorde Popovic, Amin Sadeghi, Husrev Taha Sencar, Mohammed Shinoy, Omar Sinan, Yifan Zhang, Ahmed Ali, Yassine El Kheir, Xiaosong Ma, Chaoyi Ruan

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Synthetic Data Can Mislead Evaluations: Membership Inference as Machine Text Detection

Ali Naseh, Niloofar Mireshghallah

机构 * University of Massachusettes Amherst（马萨诸塞大学阿默斯特分校） ； University of Washington（华盛顿大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Diversify, Don't Fine-Tune: Scaling Up Visual Recognition Training with Synthetic Images

Zhuoran Yu, Chenchen Zhu, Sean Culatana, Raghuraman Krishnamoorthi, Fanyi Xiao, Yong Jae Lee

机构 * University of Wisconsin-Madison（威斯康星大学麦迪逊分校） ； Meta 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted by Transactions on Machine Learning Research (TMLR)

MedGrad E-CLIP: Enhancing Trust and Transparency in AI-Driven Skin Lesion Diagnosis

Sadia Kamal, Tim Oates

机构 * University of Maryland Baltimore County（马里兰大学巴尔的摩县分校） 纠错

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments Accepted to 2025 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW)

Addressing speaker gender bias in large scale speech translation systems

Shubham Bansal, Vikas Joshi, Harveen Chadha, Rupeshkumar Mehta, Jinyu Li

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Scaling-laws for Large Time-series Models

Thomas D. P. Edwards, James Alvey, Justin Alsing, Nam H. Nguyen, Benjamin D. Wandelt

机构 * Johns Hopkins University（约翰斯·霍普金斯大学） ； University of Cambridge（剑桥大学） ； University of Amsterdam（阿姆斯特丹大学） ； Stockholm University（斯德哥尔摩大学） ； Calda AI（卡尔达人工智能公司） ； Capital One（第一资本金融公司） ； Institut d’Astrophysique de Paris（巴黎天体物理研究所） ； Flatiron Institute（弗拉蒂伦研究所） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 4 main pages (16 total), 4 figures; Accepted for oral presentation in Time Series in the Age of Large Models (TSALM) Workshop at Neurips 2024

Advancing Pancreatic Cancer Prediction with a Next Visit Token Prediction Head on top of Med-BERT

Jianping He, Laila Rasmy, Degui Zhi, Cui Tao

专题命中 预训练与数据 ：foundation model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

BaiJia: A Large-Scale Role-Playing Agent Corpus of Chinese Historical Characters

Ting Bai, Jiazheng Kang, Jiayang Fan

机构 * Beijing University of Posts and Telecommunications（北京邮电大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Unsupervised Text Embedding Space Generation Using Generative Adversarial Networks for Text Synthesis

Jun-Min Lee, Tae-Bin Ha

机构 * Korea Advanced Institute of Science Technology（韩国科学技术院） ； I-BRICKS 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments NEJLT accpeted

CLIP-RLDrive: Human-Aligned Autonomous Driving via CLIP-Based Reward Shaping in Reinforcement Learning

Erfan Doroudian, Hamid Taghavifar

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Phi-4 Technical Report

Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, Yi Zhang

专题命中 预训练与数据 ：language model(abstract)；post-training(abstract)；分类 cs.CL、cs.AI

Euclid: Supercharging Multimodal LLMs with Synthetic High-Fidelity Visual Descriptions

Jiarui Zhang, Ollie Liu, Tianyu Yu, Jinyi Hu, Willie Neiswanger

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 33 pages, 22 figures, 5 tables, 7 algorithms

Length Optimization in Conformal Prediction

Shayan Kiyani, George Pappas, Hamed Hassani

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement

Pranjal Aggarwal, Bryan Parno, Sean Welleck

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective

Kaiyue Wen, Zhiyuan Li, Jason Wang, David Hall, Percy Liang, Tengyu Ma

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 45 pages,13 figures

Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?

Jonathan Hayase, Alisa Liu, Yejin Choi, Sewoong Oh, Noah A. Smith

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments NeurIPS camera-ready, code at https://github.com/alisawuffles/tokenizer-attack

The Moral Foundations Weibo Corpus

Renjie Cao, Miaoyan Hu, Jiahan Wei, Baha Ihnaini

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

From Unstructured Data to In-Context Learning: Exploring What Tasks Can Be Learned and When

Kevin Christian Wibisono, Yixin Wang

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 39 pages

Adaptive Length Image Tokenization via Recurrent Allocation

Shivam Duggal, Phillip Isola, Antonio Torralba, William T. Freeman

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Code at: https://github.com/ShivamDuggal4/adaptive-length-tokenizer

Learning Versatile Skills with Curriculum Masking

Yao Tang, Zhihui Xie, Zichuan Lin, Deheng Ye, Shuai Li

专题命中 预训练与数据 ：pretraining(abstract)；prompting(abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2024 poster, 21 pages, 8 figures