语言大模型 / LLM

VietASR: Achieving Industry-level Vietnamese ASR with 50-hour labeled data and Large-Scale Speech Pretraining

Jianheng Zhuo, Yifan Yang, Yiwen Shao, Yong Xu, Dong Yu, Kai Yu, Xie Chen

机构 * School of Computer Science, MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University, shanghai innovation instituteChina Tencent AI LabUSA（计算机科学学院、人工智能国家重点实验室、上海交通大学、上海创新研究院中国腾讯AI实验室美国） 纠错

专题命中 预训练与数据 ：pretraining(title)；分类 cs.CL、cs.AI

Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo

Zachary Charles, Gabriel Teston, Lucio Dery, Keith Rush, Nova Fallen, Zachary Garrett, Arthur Szlam, Arthur Douillard

专题命中 预训练与数据 ：language model(title)；分类 cs.CL、cs.LG

Independence Tests for Language Models

Sally Zhu, Ahmed Ahmed, Rohith Kuditipudi, Percy Liang

专题命中 预训练与数据 ：language model(title)；分类 cs.CL、cs.LG

Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Model

Jiarui Jin, Haoyu Wang, Hongyan Li, Jun Li, Jiahui Pan, Shenda Hong

专题命中 预训练与数据 ：language model(title)；分类 cs.AI、cs.LG

Comments 21 pages, 8 figures, accepted by International Conference on Learning Representations 2025

Resolving Discrepancies in Compute-Optimal Scaling of Language Models

Tomer Porian, Mitchell Wortsman, Jenia Jitsev, Ludwig Schmidt, Yair Carmon

专题命中 预训练与数据 ：language model(title)；分类 cs.CL、cs.LG

Comments Spotlight at NeurIPS 2024

Mitigating Knowledge Conflicts in Language Model-Driven Question Answering

Han Cao, Zhaoyang Zhang, Xiangtian Li, Chufan Wu, Hansong Zhang, Wenqing Zhang

专题命中 预训练与数据 ：language model(title)；分类 cs.CL、cs.AI

Comments revised version, more figures

FGBERT: Function-Driven Pre-trained Gene Language Model for Metagenomics

ChenRui Duan, Zelin Zang, Yongjie Xu, Hang He, Zihan Liu, Siyuan Li, Zijia Song, Ju-Sheng Zheng, Stan Z. Li

专题命中 预训练与数据 ：language model(title)；分类 cs.AI、cs.LG

LDP: Generalizing to Multilingual Visual Information Extraction by Language Decoupled Pretraining

Huawen Shen, Gengluo Li, Jinwen Zhong, Yu Zhou

专题命中 预训练与数据 ：pretraining(title)；分类 cs.CL、cs.LG

Comments Accepted by AAAI2025

Open-Amp: Synthetic Data Framework for Audio Effect Foundation Models

Alec Wright, Alistair Carson, Lauri Juvela

专题命中 预训练与数据 ：foundation model(title)；分类 cs.AI、cs.LG

Looking Inward: Language Models Can Learn About Themselves by Introspection

Felix J Binder, James Chua, Tomek Korbak, Henry Sleight, John Hughes, Robert Long, Ethan Perez, Miles Turpin, Owain Evans

专题命中 预训练与数据 ：language model(title)；分类 cs.CL、cs.AI

Comments 15 pages, 9 figures

MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understanding

Qinzhuo Wu, Weikai Xu, Wei Liu, Tao Tan, Jianfeng Liu, Ang Li, Jian Luan, Bin Wang, Shuo Shang

专题命中 预训练与数据 ：language model(title)；分类 cs.CL、cs.AI

GFlowNet Pretraining with Inexpensive Rewards

Mohit Pandey, Gopeshh Subbaraj, Emmanuel Bengio

专题命中 预训练与数据 ：pretraining(title)；分类 cs.AI、cs.LG

Weakly Supervised Pretraining and Multi-Annotator Supervised Finetuning for Facial Wrinkle Detection

Ik Jun Moon, Junho Moon, Ikbeom Jang

专题命中 预训练与数据 ：pretraining(title)；分类 cs.AI、cs.LG

Single Character Perturbations Break LLM Alignment

Leon Lin, Hannah Brown, Kenji Kawaguchi, Michael Shieh

专题命中 预训练与数据 ：LLM(title)；分类 cs.CL、cs.LG

Comments 8 pages, 6 figures

Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining

Qi Zhang, Tianqi Du, Haotian Huang, Yifei Wang, Yisen Wang

专题命中 预训练与数据 ：pretraining(title)；分类 cs.CL、cs.LG

Towards Transparency: Exploring LLM Trainings Datasets through Visual Topic Modeling and Semantic Frame

Charles de Dampierre, Andrei Mogoutov, Nicolas Baumard

专题命中 预训练与数据 ：LLM(title)；分类 cs.CL、cs.AI

ChatLang-8: An LLM-Based Synthetic Data Generation Framework for Grammatical Error Correction

Jeiyoon Park, Chanjun Park, Heuiseok Lim

专题命中 预训练与数据 ：LLM(title)；分类 cs.CL、cs.AI

Comments preprint

Federated Generative Learning with Foundation Models

Jie Zhang, Xiaohua Qi, Bo Zhao

专题命中 预训练与数据 ：foundation model(title)；分类 cs.AI、cs.LG

DiffRoll: Diffusion-based Generative Music Transcription with Unsupervised Pretraining Capability

Kin Wai Cheuk, Ryosuke Sawata, Toshimitsu Uesaka, Naoki Murata, Naoya Takahashi, Shusuke Takahashi, Dorien Herremans, Yuki Mitsufuji

专题命中 预训练与数据 ：pretraining(title)；分类 cs.AI、cs.LG

Journal ref Proceedings of ICASSP - IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023

HiGPT: Heterogeneous Graph Language Model

Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Long Xia, Dawei Yin, Chao Huang

专题命中 预训练与数据 ：language model(title)；分类 cs.CL、cs.LG

Comments Accepted by KDD'2024, full paper

Persian Pronoun Resolution: Leveraging Neural Networks and Language Models

Hassan Haji Mohammadi, Alireza Talebpour, Ahmad Mahmoudi Aznaveh, Samaneh Yazdani

专题命中 预训练与数据 ：language model(title)；分类 cs.CL、cs.AI

Using Generative Text Models to Create Qualitative Codebooks for Student Evaluations of Teaching

Andrew Katz, Mitchell Gerhardt, Michelle Soledad

专题命中 预训练与数据 ：large language model(abstract,comments)；language model(abstract,comments)；分类 cs.CL、cs.AI

Comments Natural language processing, large language models, generative AI, student evaluations of teaching, codebook generation, qualitative data analysis

KAUCUS: Knowledge Augmented User Simulators for Training Language Model Assistants

Kaustubh D. Dhole

专题命中 预训练与数据 ：language model(title)；分类 cs.CL、cs.AI

Comments Simulation of Conversational Intelligence in Chat, EACL 2024

Siamese Networks with Soft Labels for Unsupervised Lesion Detection and Patch Pretraining on Screening Mammograms

Kevin Van Vorst, Li Shen

专题命中 预训练与数据 ：pretraining(title)；分类 cs.AI、cs.LG

PLEX: Making the Most of the Available Data for Robotic Manipulation Pretraining

Garrett Thomas, Ching-An Cheng, Ricky Loynd, Felipe Vieira Frujeri, Vibhav Vineet, Mihai Jalobeanu, Andrey Kolobov

专题命中 预训练与数据 ：pretraining(title)；分类 cs.AI、cs.LG

CHIP: Contrastive Hierarchical Image Pretraining

Arpit Mittal, Harshil Jhaveri, Swapnil Mallick, Abhishek Ajmera

专题命中 预训练与数据 ：pretraining(title)；分类 cs.AI、cs.LG

DiffNAS: Bootstrapping Diffusion Models by Prompting for Better Architectures

Wenhao Li, Xiu Su, Shan You, Fei Wang, Chen Qian, Chang Xu

专题命中 预训练与数据 ：prompting(title)；分类 cs.AI、cs.LG

Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining?

Fei Wang, Liang Ding, Jun Rao, Ye Liu, Li Shen, Changxing Ding

专题命中 预训练与数据 ：pretraining(title)；分类 cs.CL、cs.AI

Comments [TL;DR] we design and release the SNARE, the first large-scale multimodal alignment probing benchmark for current vision-language pretrained models

Comparison of Pre-trained Language Models for Turkish Address Parsing

Muhammed Cihat Ünal, Betül Aygün, Aydın Gerek

专题命中 预训练与数据 ：language model(title)；分类 cs.CL、cs.LG

Comments published in 16th UYMS (2023) https://ekitap.atauni.edu.tr/index.php/product/16-ulusal-yazilim-muhendisligi-sempozyumu-bildiri-kitabi/

Journal ref Ulusal Yazılım Mühendisliği Sempozyumu, 16, 155-165 (Erzurum 2023)

Retrieval-Augmented Multimodal Language Modeling

Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, Wen-tau Yih

专题命中 预训练与数据 ：language model(title)；分类 cs.CL、cs.LG

Comments Published at ICML 2023. Blog post available at https://cs.stanford.edu/~myasu/blog/racm3/