语言大模型 / LLM - arXivDaily 专题

2409.14494 2025-03-13 cs.CL cs.LG cs.SD eess.AS 62%

CPT-Boosted Wav2vec2.0: Towards Noise Robust Speech Recognition for Classroom Environments

Ahmed Adel Attia, Dorottya Demszky, Tolulope Ogunremi, Jing Liu, Carol Espy-Wilson

机构 * University of Maryland（马里兰大学）； Stanford University（斯坦福大学）

专题命中预训练与数据：pretraining(abstract)；分类 cs.CL、cs.LG

Comments arXiv admin note: substantial text overlap with arXiv:2405.13018

Journal ref ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

2502.04684 2025-03-11 cs.LG cs.AI 62%

G2PDiffusion: Cross-Species Genotype-to-Phenotype Prediction via Evolutionary Diffusion

Mengdi Liu, Zhangyang Gao, Hong Chang, Stan Z. Li, Shiguang Shan, Xilin Chen

机构 * University of Chinese Academy of Sciences（中国科学院大学）； AI Lab, Research Center for Industries of the Future, Westlake University（西湖大学未来产业研究中心AI实验室）

专题命中预训练与数据：pretraining(abstract)；分类 cs.AI、cs.LG

2412.04532 2025-03-11 cs.LG cs.AI 62%

WinTSR: A Windowed Temporal Saliency Rescaling Method for Interpreting Time Series Deep Learning Models

Md. Khairul Islam, Judy Fox

专题命中预训练与数据：foundation model(abstract)；分类 cs.AI、cs.LG

Comments 11 pages, 14 figures, GitHub https://github.com/khairulislam/Timeseries-Explained

2503.03172 2025-03-06 cs.CL cs.AI 62%

Intermediate-Task Transfer Learning: Leveraging Sarcasm Detection for Stance Detection

Gibson Nkhata, Susan Gauch

机构 * University of Arkansas（阿肯色大学）

专题命中预训练与数据：pretraining(abstract)；分类 cs.CL、cs.AI

Comments 8 pages, 2 figures, published in The Sixteenth International Conference on Information (eKNOW 2024)

2503.02057 2025-03-05 cs.CL cs.AI q-bio.NC 62%

Hebbian learning the local structure of language

P. Myles Eugenio

机构 * Indiana University（印第安纳大学）； University of Connecticut（康涅狄格大学）； Harvard University（哈佛大学）

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.AI

Comments 10 figures, 14 pages

2410.23825 2025-03-05 cs.CL cs.AI 62%

GlotCC: An Open Broad-Coverage CommonCrawl Corpus and Pipeline for Minority Languages

Amir Hossein Kargaran, François Yvon, Hinrich Schütze

机构 * LMU Munich（慕尼黑大学）； Munich Center for Machine Learning（慕尼黑机器学习中心）； Sorbonne Université（索邦大学）； CNRS（法国国家科学研究中心）； ISIR（巴黎机器人与智能系统研究所）

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024

2402.18945 2025-03-04 cs.CR cs.AI cs.CL 62%

SynGhost: Invisible and Universal Task-agnostic Backdoor Attack via Syntactic Transfer

Pengzhou Cheng, Wei Du, Zongru Wu, Fengwei Zhang, Libo Chen, Zhuosheng Zhang, Gongshen Liu

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 16 figures, 12 tables, accepted at NAACL 2025 Findings

2502.18249 2025-02-26 cs.LG cs.CL cs.IT math.IT 62%

Iterative Counterfactual Data Augmentation

Mitchell Plyler, Min Chi

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.LG

Comments AAAI 2025

2502.17390 2025-02-25 cs.CL cs.LG 62%

Mitigating Bias in RAG: Controlling the Embedder

Taeyoun Kim, Jacob Springer, Aditi Raghunathan, Maarten Sap

专题命中预训练与数据：LLM(abstract)；分类 cs.CL、cs.LG

Comments 26 pages (8 main), 12 figures, 7 tables

2502.15950 2025-02-25 cs.LG cs.CL 62%

Optimizing Pre-Training Data Mixtures with Mixtures of Data Expert Models

Lior Belenki, Alekh Agarwal, Tianze Shi, Kristina Toutanova

机构 * Google DeepMind（谷歌DeepMind）； Google Research（谷歌研究院）

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.LG

2502.14620 2025-02-21 cs.CL cs.AI 62%

Exploring RWKV for Sentence Embeddings: Layer-wise Analysis and Baseline Comparison for Semantic Similarity

Xinghan Pan

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 3 tables, preprint on ArXiV, includes detailed analysis of RWKV for semantic similarity tasks

2405.17139 2025-02-18 cs.CV cs.AI cs.LG 62%

Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling

Cristian Rodriguez-Opazo, Ehsan Abbasnejad, Damien Teney, Hamed Damirchi, Edison Marrese-Taylor, Anton van den Hengel

专题命中预训练与数据：pretraining(abstract)；分类 cs.AI、cs.LG

Comments ICLR 2025. arXiv admin note: text overlap with arXiv:2312.14400

2502.03824 2025-02-17 cs.CL cs.AI 62%

Syntriever: How to Train Your Retriever with Synthetic Data from LLMs

Minsang Kim, Seungjun Baek

机构 * Korea University（高丽大学）； SK Telecom（SK电信）

专题命中预训练与数据：LLM(abstract)；分类 cs.CL、cs.AI

Comments the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL), Findings, Accepted

2502.05937 2025-02-11 cs.CL cs.AI 62%

A Semi-Supervised Text Generation Framework Combining a Deep Transformer and a GAN

Shengquan Wang

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.AI

Comments 7 pages

2408.10189 2025-02-11 cs.LG cs.AI 62%

Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models

Aviv Bick, Kevin Y. Li, Eric P. Xing, J. Zico Kolter, Albert Gu

专题命中预训练与数据：language model(abstract)；分类 cs.AI、cs.LG

2502.04351 2025-02-10 cs.CL cs.AI 62%

NER4all or Context is All You Need: Using LLMs for low-effort, high-performance NER on historical texts. A humanities informed approach

Torsten Hiltmann, Martin Dröge, Nicole Dresselhaus, Till Grallert, Melanie Althage, Paul Bayer, Sophie Eckenstaler, Koray Mendi, Jascha Marijn Schmitz, Philipp Schneider, Wiebke Sczeponik, Anica Skibba

专题命中预训练与数据：prompting(abstract)；分类 cs.CL、cs.AI

2406.03857 2025-02-07 cs.LG cs.CL cs.CV 62%

MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognition

Stefan Gerd Fritsch, Cennet Oguz, Vitor Fortes Rey, Lala Ray, Maximilian Kiefer-Emmanouilidis, Paul Lukowicz

机构 * DFKI（德国人工智能研究中心）； RPTU Kaiserslautern-Landau（凯撒斯劳滕-兰道莱茵-普法尔茨大学）

专题命中预训练与数据：foundation model(abstract)；分类 cs.CL、cs.LG

2502.03128 2025-02-06 cs.SD cs.AI cs.LG eess.AS eess.SP 62%

Metis: A Foundation Speech Generation Model with Masked Generative Pre-training

Yuancheng Wang, Jiachen Zheng, Junan Zhang, Xueyao Zhang, Huan Liao, Zhizheng Wu

机构 * The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））

专题命中预训练与数据：foundation model(abstract)；分类 cs.AI、cs.LG

2202.08835 2025-01-17 cs.LG cs.AI cs.CV stat.ML 62%

General Cyclical Training of Neural Networks

Leslie N. Smith

专题命中预训练与数据：pretraining(abstract)；分类 cs.AI、cs.LG

Comments Position paper

Journal ref AAIML.2023.1157

2410.05450 2025-01-14 cs.CV cs.AI cs.LG 62%

AI-Driven Early Mental Health Screening: Analyzing Selfies of Pregnant Women

Gustavo A. Basílio, Thiago B. Pereira, Alessandro L. Koerich, Hermano Tavares, Ludmila Dias, Maria das Graças da S. Teixeira, Rafael T. Sousa, Wilian H. Hisatugu, Amanda S. Mota, Anilton S. Garcia, Marco Aurélio K. Galletta, Thiago M. Paixão

机构 * Federal Institute of Espírito Santo (IFES)（圣埃斯皮里图联邦学院）； École de Technologie Supérieure (ÉTS)（高等技术学院）； University of São Paulo Medical School (FMUSP)（圣保罗大学医学院）； Federal University of Espírito Santo (UFES)（圣埃斯皮里图联邦大学）； Federal University of Mato Grosso (UFMT)（马托格罗索联邦大学）

专题命中预训练与数据：language model(abstract)；分类 cs.AI、cs.LG

Comments This article has been accepted for publication in HEALTHINF25 at the 18th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2025)

2410.02780 2025-01-13 cs.CV cs.AI cs.LG 62%

Guess What I Think: Streamlined EEG-to-Image Generation with Latent Diffusion Models

Eleonora Lopez, Luigi Sigillo, Federica Colonnese, Massimo Panella, Danilo Comminiello

机构 * Sapienza University of Rome（罗马大学）

专题命中预训练与数据：pretraining(abstract)；分类 cs.AI、cs.LG

Comments Accepted at ICASSP 2025

2501.02156 2025-01-09 cs.LG cs.AI cs.PF 62%

The Race to Efficiency: A New Perspective on AI Scaling Laws

Chien-Ping Lu

机构 * Nimbyss

专题命中预训练与数据：prompting(abstract)；分类 cs.AI、cs.LG

Comments 21 pages, 3 figures. 2 tables, second draft

2501.04155 2025-01-09 cs.CV cs.CL cs.LG 62%

MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curation

Siddharth Joshi, Besmira Nushi, Vidhisha Balachandran, Varun Chandrasekaran, Vibhav Vineet, Neel Joshi, Baharan Mirzasoleiman

机构 * Microsoft Research（微软研究院）； UCLA（加利福尼亚大学洛杉矶分校）； UIUC（伊利诺伊大学厄巴纳-香槟分校）

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.LG

2501.04153 2025-01-09 cs.CL cs.LG 62%

Multilingual Open QA on the MIA Shared Task

Navya Yarrabelly, Saloni Mittal, Ketan Todi, Kimihiro Hasegawa

机构 * Language Technologies Institute（语言技术研究所）

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.LG

2501.03540 2025-01-08 cs.LG cs.AI 62%

Deep Learning within Tabular Data: Foundations, Challenges, Advances and Future Directions

Weijieying Ren, Tianxiang Zhao, Yuqing Huang, Vasant Honavar

机构 * The Pennsylvania State University（宾夕法尼亚州立大学）； University of Science and Technology of China（中国科学技术大学）

专题命中预训练与数据：foundation model(abstract)；分类 cs.AI、cs.LG

2412.10182 2025-01-07 cs.CV cs.AI cs.LG 62%

Multi-Head Encoding for Extreme Label Classification

Daojun Liang, Haixia Zhang, Dongfeng Yuan, Minggao Zhang

机构 * Shandong University（山东大学）； School of Information Science and Engineering, Shandong University（山东大学信息科学与工程学院）； School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）； School of Qilu Transportation, Shandong University（山东大学齐鲁交通学院）

专题命中预训练与数据：pretraining(abstract)；分类 cs.AI、cs.LG

Comments 20 pages, 12 figs, Published in IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2024

2501.01069 2025-01-03 cs.CL cs.LG 62%

BeliN: A Novel Corpus for Bengali Religious News Headline Generation using Contextual Feature Fusion

Md Osama, Ashim Dey, Kawsar Ahmed, Muhammad Ashad Kabir

机构 * Chittagong University of Engineering and Technology（吉大港工程技术大学）； Bangladesh Army University of Science and Technology (BAUST)（孟加拉国陆军科技大学）； Charles Sturt University（查尔斯斯特大学）

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.LG

Comments 28 pages, 4 figures, 11 tables

2411.17470 2025-01-03 cs.CV cs.AI cs.LG 62%

Towards Precise Scaling Laws for Video Diffusion Transformers

Yuanyang Yin, Yaqi Zhao, Mingwu Zheng, Ke Lin, Jiarong Ou, Rui Chen, Victor Shea-Jay Huang, Jiahao Wang, Xin Tao, Pengfei Wan, Di Zhang, Baoqun Yin, Wentao Zhang, Kun Gai

机构 * University of Science and Technology of China（中国科学技术大学）； Peking University（北京大学）； Kuaishou Technology（快手科技）

专题命中预训练与数据：language model(abstract)；分类 cs.AI、cs.LG

2412.18917 2024-12-30 cs.CV cs.AI cs.LG 62%

Open-Vocabulary Panoptic Segmentation Using BERT Pre-Training of Vision-Language Multiway Transformer Model

Yi-Chia Chen, Wei-Hua Li, Chu-Song Chen

机构 * National Taiwan University（台湾大学）

专题命中预训练与数据：foundation model(abstract)；分类 cs.AI、cs.LG

Comments ICIP 2024

2402.02262 2024-12-23 cs.CL cs.AI 62%

Data Quality Matters: Suicide Intention Detection on Social Media Posts Using RoBERTa-CNN

Emily Lin, Jian Sun, Hsingyu Chen, Mohammad H. Mahoor

机构 * University of Denver（丹佛大学）

专题命中预训练与数据：pretraining(abstract)；分类 cs.CL、cs.AI

Comments 4 pages, 1 figure, 4 tables

Journal ref 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 2024, pp. 1-5

1. 预训练与数据 12502 篇

CPT-Boosted Wav2vec2.0: Towards Noise Robust Speech Recognition for Classroom Environments

G2PDiffusion: Cross-Species Genotype-to-Phenotype Prediction via Evolutionary Diffusion

WinTSR: A Windowed Temporal Saliency Rescaling Method for Interpreting Time Series Deep Learning Models

Intermediate-Task Transfer Learning: Leveraging Sarcasm Detection for Stance Detection

Hebbian learning the local structure of language

GlotCC: An Open Broad-Coverage CommonCrawl Corpus and Pipeline for Minority Languages

SynGhost: Invisible and Universal Task-agnostic Backdoor Attack via Syntactic Transfer

Iterative Counterfactual Data Augmentation

Mitigating Bias in RAG: Controlling the Embedder

Optimizing Pre-Training Data Mixtures with Mixtures of Data Expert Models

Exploring RWKV for Sentence Embeddings: Layer-wise Analysis and Baseline Comparison for Semantic Similarity

Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling

Syntriever: How to Train Your Retriever with Synthetic Data from LLMs

A Semi-Supervised Text Generation Framework Combining a Deep Transformer and a GAN

Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models

NER4all or Context is All You Need: Using LLMs for low-effort, high-performance NER on historical texts. A humanities informed approach

MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognition

Metis: A Foundation Speech Generation Model with Masked Generative Pre-training

General Cyclical Training of Neural Networks

AI-Driven Early Mental Health Screening: Analyzing Selfies of Pregnant Women

Guess What I Think: Streamlined EEG-to-Image Generation with Latent Diffusion Models

The Race to Efficiency: A New Perspective on AI Scaling Laws

MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curation

Multilingual Open QA on the MIA Shared Task

Deep Learning within Tabular Data: Foundations, Challenges, Advances and Future Directions

Multi-Head Encoding for Extreme Label Classification

BeliN: A Novel Corpus for Bengali Religious News Headline Generation using Contextual Feature Fusion

Towards Precise Scaling Laws for Video Diffusion Transformers

Open-Vocabulary Panoptic Segmentation Using BERT Pre-Training of Vision-Language Multiway Transformer Model

Data Quality Matters: Suicide Intention Detection on Social Media Posts Using RoBERTa-CNN