语言大模型 / LLM - arXivDaily 专题

2310.02994 2024-12-12 cs.LG cs.AI stat.ML 84%

Multiple Physics Pretraining for Physical Surrogate Models

Michael McCabe, Bruno Régaldo-Saint Blancard, Liam Holden Parker, Ruben Ohana, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Siavash Golkar, Geraud Krawezik, Francois Lanusse, Mariel Pettee, Tiberiu Tesileanu, Kyunghyun Cho, Shirley Ho

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；分类 cs.AI、cs.LG

2305.08339 2024-12-10 cs.CL cs.AI 84%

Assessing the potential of LLM-assisted annotation for corpus-based pragmatics and discourse analysis: The case of apology

Danni Yu, Luyang Li, Hang Su, Matteo Fuoli

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 24 pages, 2 figures, 3 tablels

2410.15661 2024-12-10 cs.CL cs.LG 84%

Scalable Data Ablation Approximations for Language Models through Modular Training and Merging

Clara Na, Ian Magnusson, Ananya Harsh Jha, Tom Sherborne, Emma Strubell, Jesse Dodge, Pradeep Dasigi

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments EMNLP 2024. 17 pages

2408.14471 2024-12-09 cs.CV cs.CL cs.LG 84%

A Practitioner's Guide to Continual Multimodal Pretraining

Karsten Roth, Vishaal Udandarao, Sebastian Dziadzio, Ameya Prabhu, Mehdi Cherti, Oriol Vinyals, Olivier Hénaff, Samuel Albanie, Matthias Bethge, Zeynep Akata

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；分类 cs.CL、cs.LG

Comments Technical Report. 52 pages. Shorter version published at the NeurIPS 2024 Dataset & Benchmarks track

2410.12883 2024-12-05 cs.CL cs.LG 84%

Scaling Laws for Multilingual Language Models

Yifei He, Alon Benhaim, Barun Patra, Praneetha Vaddamanu, Sanchit Ahuja, Parul Chopra, Vishrav Chaudhary, Han Zhao, Xia Song

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

2411.15099 2024-11-25 cs.CV cs.CL cs.LG 84%

Context-Aware Multimodal Pretraining

Karsten Roth, Zeynep Akata, Dima Damen, Ivana Balažević, Olivier J. Hénaff

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

2406.06046 2024-11-19 cs.CL cs.LG 84%

MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models

Zichun Yu, Spandan Das, Chenyan Xiong

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted to NeurIPS 2024

2411.03356 2024-11-07 cs.LG cs.AI 84%

Enhancing Table Representations with LLM-powered Synthetic Data Generation

Dayu Yang, Natawut Monaikul, Amanda Ding, Bozhao Tan, Kishore Mosaliganti, Giri Iyengar

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments the Thirty-Eighth Annual Conference on Neural Information Processing Systems Table Representation Workshop

2405.17992 2024-11-05 cs.CL cs.AI q-bio.NC 84%

fMRI predictors based on language models of increasing complexity recover brain left lateralization

Laurent Bonnasse-Gahot, Christophe Pallier

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

2411.00997 2024-11-05 cs.CV cs.AI cs.CL cs.CY 84%

Identifying Implicit Social Biases in Vision-Language Models

Kimia Hamidieh, Haoran Zhang, Walter Gerych, Thomas Hartvigsen, Marzyeh Ghassemi

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

2410.07018 2024-11-05 cs.LG cs.AI 84%

Tri-Level Navigator: LLM-Empowered Tri-Level Learning for Time Series OOD Generalization

Chengtao Jian, Kai Yang, Yang Jiao

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted at NeurIPS 2024

2405.16528 2024-11-05 cs.LG cs.CL 84%

LoQT: Low-Rank Adapters for Quantized Pretraining

Sebastian Loeschcke, Mads Toftrup, Michael J. Kastoryano, Serge Belongie, Vésteinn Snæbjarnarson

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

2410.24177 2024-11-01 eess.AS cs.CL cs.LG cs.SD 84%

DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Models

Heng-Jui Chang, Hongyu Gong, Changhan Wang, James Glass, Yu-An Chung

专题命中预训练与数据：language model(title,abstract)；SLM(abstract)；分类 cs.CL、cs.LG

Comments Preprint

2410.18160 2024-10-25 cs.CL cs.LG 84%

Future Token Prediction -- Causal Language Modelling with Per-Token Semantic State Vector for Multi-Token Prediction

Nicholas Walker

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 15 pages, 7 figures, 3 tables

2410.12294 2024-10-18 cs.HC cs.CL cs.CY cs.LG 84%

LLM-based Cognitive Models of Students with Misconceptions

Shashank Sonkar, Xinghe Chen, Naiming Liu, Richard G. Baraniuk, Mrinmaya Sachan

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2410.08612 2024-10-14 cs.CV cs.AI cs.LG 84%

Synth-SONAR: Sonar Image Synthesis with Enhanced Diversity and Realism via Dual Diffusion Models and GPT Prompting

Purushothaman Natarajan, Kamal Basha, Athira Nambiar

专题命中预训练与数据：prompting(title,abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 12 pages, 5 tables and 9 figures

2410.03884 2024-10-08 cs.CL cs.AI cs.CY cs.HC 84%

KidLM: Advancing Language Models for Children -- Early Insights and Future Directions

Mir Tafseer Nayeem, Davood Rafiei

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2024 (long, main)

2408.04057 2024-10-04 cs.LG cs.AI 84%

PowerPM: Foundation Model for Power Systems

Shihao Tu, Yupeng Zhang, Jing Zhang, Zhendong Fu, Yin Zhang, Yang Yang

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments 23 pages, 5 figures, 8 tables

2311.09210 2024-10-04 cs.CL cs.AI 84%

Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models

Wenhao Yu, Hongming Zhang, Xiaoman Pan, Kaixin Ma, Hongwei Wang, Dong Yu

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024 (main conference)

2410.00179 2024-10-03 cs.CL cs.LG 84%

Evaluating the fairness of task-adaptive pretraining on unlabeled test data before few-shot text classification

Kush Dubey

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments To appear in the GenBench Workshop at EMNLP 2024

2409.14705 2024-09-24 cs.CL cs.AI 84%

Target-Aware Language Modeling via Granular Data Sampling

Ernie Chang, Pin-Jie Lin, Yang Li, Changsheng Zhao, Daeil Kim, Rastislav Rabatin, Zechun Liu, Yangyang Shi, Vikas Chandra

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2024 Main Conference, 9 pages, 6 figures, 3 tables

2409.08111 2024-09-13 cs.LG cs.AI cs.CR cs.NI 84%

Towards a graph-based foundation model for network traffic analysis

Louis Van Langendonck, Ismael Castell-Uroz, Pere Barlet-Ros

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments Pre-print of Accepted Workshop paper to 3rd GNNet, co-located with CoNEXT'24

2409.02326 2024-09-05 cs.CL cs.AI 84%

Arctic-SnowCoder: Demystifying High-Quality Data in Code Pretraining

Yuxiang Wei, Hojae Han, Rajhans Samdani

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI

2408.09757 2024-08-20 cs.LG cs.CL cs.CY 84%

Strategic Demonstration Selection for Improved Fairness in LLM In-Context Learning

Jingyu Hu, Weiru Liu, Mengnan Du

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2304.05215 2024-08-13 cs.CV cs.AI cs.LG 84%

A Billion-scale Foundation Model for Remote Sensing Images

Keumgang Cha, Junghoon Seo, Taekyung Lee

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments This manuscript is the accepted version for IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (IEEE J-STARS)

2405.12630 2024-08-12 cs.CL cs.AI 84%

Exploration of Masked and Causal Language Modelling for Text Generation

Nicolo Micheletti, Samuel Belkadi, Lifeng Han, Goran Nenadic

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments working paper - under review

2402.11782 2024-08-12 cs.CL cs.LG 84%

What Evidence Do Language Models Find Convincing?

Alexander Wan, Eric Wallace, Dan Klein

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments ACL 2024 (Main)

2403.19154 2024-08-09 cs.CL cs.AI 84%

STaR-GATE: Teaching Language Models to Ask Clarifying Questions

Chinmaya Andukuri, Jan-Philipp Fränken, Tobias Gerstenberg, Noah D. Goodman

专题命中预训练与数据：language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

2308.01154 2024-08-05 cs.AI cs.CL 84%

Arithmetic with Language Models: from Memorization to Computation

Davide Maltoni, Matteo Ferrara

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments The article has been accepted for publication in Elsevier Neural Networks journal. The final version is available on the Elsevier ScienceDirect platform

Journal ref \Neural Networks, vol. 179, 2024

2404.00614 2024-08-01 cs.CL cs.AI 84%

Learning to Plan for Language Modeling from Unlabeled Data

Nathan Cornille, Marie-Francine Moens, Florian Mai

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments Published at COLM 2024

1. 预训练与数据 12429 篇

Multiple Physics Pretraining for Physical Surrogate Models

Assessing the potential of LLM-assisted annotation for corpus-based pragmatics and discourse analysis: The case of apology

Scalable Data Ablation Approximations for Language Models through Modular Training and Merging

A Practitioner's Guide to Continual Multimodal Pretraining

Scaling Laws for Multilingual Language Models

Context-Aware Multimodal Pretraining

MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models

Enhancing Table Representations with LLM-powered Synthetic Data Generation

fMRI predictors based on language models of increasing complexity recover brain left lateralization

Identifying Implicit Social Biases in Vision-Language Models

Tri-Level Navigator: LLM-Empowered Tri-Level Learning for Time Series OOD Generalization

LoQT: Low-Rank Adapters for Quantized Pretraining

DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Models

Future Token Prediction -- Causal Language Modelling with Per-Token Semantic State Vector for Multi-Token Prediction

LLM-based Cognitive Models of Students with Misconceptions

Synth-SONAR: Sonar Image Synthesis with Enhanced Diversity and Realism via Dual Diffusion Models and GPT Prompting

KidLM: Advancing Language Models for Children -- Early Insights and Future Directions

PowerPM: Foundation Model for Power Systems

Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models

Evaluating the fairness of task-adaptive pretraining on unlabeled test data before few-shot text classification

Target-Aware Language Modeling via Granular Data Sampling

Towards a graph-based foundation model for network traffic analysis

Arctic-SnowCoder: Demystifying High-Quality Data in Code Pretraining

Strategic Demonstration Selection for Improved Fairness in LLM In-Context Learning

A Billion-scale Foundation Model for Remote Sensing Images

Exploration of Masked and Causal Language Modelling for Text Generation

What Evidence Do Language Models Find Convincing?

STaR-GATE: Teaching Language Models to Ask Clarifying Questions

Arithmetic with Language Models: from Memorization to Computation

Learning to Plan for Language Modeling from Unlabeled Data