语言大模型 / LLM

Command R7B Arabic: A Small, Enterprise Focused, Multilingual, and Culturally Aware Arabic LLM

Yazeed Alnumay, Alexandre Barbet, Anna Bialas, William Darling, Shaan Desai, Joan Devassy, Kyle Duffy, Stephanie Howe, Olivia Lasche, Justin Lee, Anirudh Shrinivason, Jennifer Tracey

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

The Power of LLM-Generated Synthetic Data for Stance Detection in Online Political Discussions

Stefan Sylvius Wagner, Maike Behrendt, Marc Ziegele, Stefan Harmeling

专题命中 预训练与数据 ：LLM(title,abstract)；prompting(abstract)；分类 cs.CL、cs.LG

Comments ICLR 2025 Spotlight

Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling

David Grangier, Simin Fan, Skyler Seto, Pierre Ablin

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 23 pages, presented at the International Conference on Learning Representation (ICLR), 2025

Hints-In-Browser: Benchmarking Language Models for Programming Feedback Generation

Nachiket Kotalwar, Alkis Gotovos, Adish Singla

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

Scaling LLM Pre-training with Vocabulary Curriculum

Fangyuan Yu

专题命中 预训练与数据 ：LLM(title)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments under review

Krutrim LLM: Multilingual Foundational Model for over a Billion People

Aditya Kallappa, Palash Kamble, Abhinav Ravi, Akshat Patidar, Vinayak Dhruv, Deepak Kumar, Raghav Awasthi, Arveti Manjunath, Himanshu Gupta, Shubham Agarwal, Kumar Ashish, Gautam Bhargava, Chandra Khatri

专题命中 预训练与数据 ：LLM(title,abstract)；foundation model(abstract)；分类 cs.CL、cs.AI

Mitigating Data Scarcity in Time Series Analysis: A Foundation Model with Series-Symbol Data Generation

Wenxuan Wang, Kai Wu, Yujian Betterest Li, Dan Wang, Xiaoyu Zhang, Jing Liu

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Transformers and Their Roles as Time Series Foundation Models

Dennis Wu, Yihan He, Yuan Cao, Jianqing Fan, Han Liu

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments 34 Pages, 2 Figures

COS(M+O)S: Curiosity and RL-Enhanced MCTS for Exploring Story Space via Language Models

Tobias Materzok

专题命中 预训练与数据 ：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

RegMix: Data Mixture as Regression for Language Model Pre-training

Qian Liu, Xiaosen Zheng, Niklas Muennighoff, Guangtao Zeng, Longxu Dou, Tianyu Pang, Jing Jiang, Min Lin

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments ICLR 2025

Self-supervised video pretraining yields robust and more human-aligned visual representations

Nikhil Parthasarathy, S. M. Ali Eslami, João Carreira, Olivier J. Hénaff

专题命中 预训练与数据 ：pretraining(title,abstract)；foundation model(abstract)；分类 cs.AI、cs.LG

Comments Accepted to 37th Conference on Neural Information Processing Systems (NeurIPS 2023)

Whose Morality Do They Speak? Unraveling Cultural Bias in Multilingual Language Models

Meltem Aksoy

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Applying LLM and Topic Modelling in Psychotherapeutic Contexts

Alexander Vanin, Vadim Bolshev, Anastasia Panfilova

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 18 pages, 4 figures

LLMCL-GEC: Advancing Grammatical Error Correction with LLM-Driven Curriculum Learning

Tao Fang, Derek F. Wong, Lusheng Zhang, Keyan Jin, Qiang Zhang, Tianjiao Li, Jinlong Hou, Lidia S. Chao

专题命中 预训练与数据 ：LLM(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Derek F. Wong is the corresponding author. The preprint version consists of 15 Pages, 5 Figures, 5 Tables, and 3 Appendices

NoteContrast: Contrastive Language-Diagnostic Pretraining for Medical Text

Prajwal Kailas, Max Homilius, Rahul C. Deo, Calum A. MacRae

专题命中 预训练与数据 ：pretraining(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Journal ref Proceedings of the 3rd Machine Learning for Health Symposium, PMLR 225:201-216, 2023

HITgram: A Platform for Experimenting with n-gram Language Models

Shibaranjani Dasgupta, Chandan Maity, Somdip Mukherjee, Rohan Singh, Diptendu Dutta, Debasish Jana

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

SOLO: A Single Transformer for Scalable Vision-Language Modeling

Yangyi Chen, Xingyao Wang, Hao Peng, Heng Ji

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted to TMLR

Seeing Syntax: Uncovering Syntactic Learning Limitations in Vision-Language Models

Sri Harsha Dumpala, David Arps, Sageev Oore, Laura Kallmeyer, Hassan Sajjad

专题命中 预训练与数据 ：language model(title,abstract)；foundation model(abstract)；分类 cs.CL、cs.LG

Multiple Physics Pretraining for Physical Surrogate Models

Michael McCabe, Bruno Régaldo-Saint Blancard, Liam Holden Parker, Ruben Ohana, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Siavash Golkar, Geraud Krawezik, Francois Lanusse, Mariel Pettee, Tiberiu Tesileanu, Kyunghyun Cho, Shirley Ho

专题命中 预训练与数据 ：pretraining(title,abstract)；foundation model(abstract)；分类 cs.AI、cs.LG

Assessing the potential of LLM-assisted annotation for corpus-based pragmatics and discourse analysis: The case of apology

Danni Yu, Luyang Li, Hang Su, Matteo Fuoli

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 24 pages, 2 figures, 3 tablels

Scalable Data Ablation Approximations for Language Models through Modular Training and Merging

Clara Na, Ian Magnusson, Ananya Harsh Jha, Tom Sherborne, Emma Strubell, Jesse Dodge, Pradeep Dasigi

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments EMNLP 2024. 17 pages

A Practitioner's Guide to Continual Multimodal Pretraining

Karsten Roth, Vishaal Udandarao, Sebastian Dziadzio, Ameya Prabhu, Mehdi Cherti, Oriol Vinyals, Olivier Hénaff, Samuel Albanie, Matthias Bethge, Zeynep Akata

专题命中 预训练与数据 ：pretraining(title,abstract)；foundation model(abstract)；分类 cs.CL、cs.LG

Comments Technical Report. 52 pages. Shorter version published at the NeurIPS 2024 Dataset & Benchmarks track

Scaling Laws for Multilingual Language Models

Yifei He, Alon Benhaim, Barun Patra, Praneetha Vaddamanu, Sanchit Ahuja, Parul Chopra, Vishrav Chaudhary, Han Zhao, Xia Song

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Context-Aware Multimodal Pretraining

Karsten Roth, Zeynep Akata, Dima Damen, Ivana Balažević, Olivier J. Hénaff

专题命中 预训练与数据 ：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models

Zichun Yu, Spandan Das, Chenyan Xiong

专题命中 预训练与数据 ：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted to NeurIPS 2024

Enhancing Table Representations with LLM-powered Synthetic Data Generation

Dayu Yang, Natawut Monaikul, Amanda Ding, Bozhao Tan, Kishore Mosaliganti, Giri Iyengar

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments the Thirty-Eighth Annual Conference on Neural Information Processing Systems Table Representation Workshop

fMRI predictors based on language models of increasing complexity recover brain left lateralization

Laurent Bonnasse-Gahot, Christophe Pallier

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

Identifying Implicit Social Biases in Vision-Language Models

Kimia Hamidieh, Haoran Zhang, Walter Gerych, Thomas Hartvigsen, Marzyeh Ghassemi

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Tri-Level Navigator: LLM-Empowered Tri-Level Learning for Time Series OOD Generalization

Chengtao Jian, Kai Yang, Yang Jiao

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted at NeurIPS 2024

LoQT: Low-Rank Adapters for Quantized Pretraining

Sebastian Loeschcke, Mads Toftrup, Michael J. Kastoryano, Serge Belongie, Vésteinn Snæbjarnarson

专题命中 预训练与数据 ：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG