语言大模型 / LLM - arXivDaily 专题

2502.14620 2025-02-21 cs.CL cs.AI 62%

Exploring RWKV for Sentence Embeddings: Layer-wise Analysis and Baseline Comparison for Semantic Similarity

Xinghan Pan

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 3 tables, preprint on ArXiV, includes detailed analysis of RWKV for semantic similarity tasks

2405.17139 2025-02-18 cs.CV cs.AI cs.LG 62%

Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling

Cristian Rodriguez-Opazo, Ehsan Abbasnejad, Damien Teney, Hamed Damirchi, Edison Marrese-Taylor, Anton van den Hengel

专题命中预训练与数据：pretraining(abstract)；分类 cs.AI、cs.LG

Comments ICLR 2025. arXiv admin note: text overlap with arXiv:2312.14400

2502.03824 2025-02-17 cs.CL cs.AI 62%

Syntriever: How to Train Your Retriever with Synthetic Data from LLMs

Minsang Kim, Seungjun Baek

机构 * Korea University（高丽大学）； SK Telecom（SK电信）

专题命中预训练与数据：LLM(abstract)；分类 cs.CL、cs.AI

Comments the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL), Findings, Accepted

2502.05937 2025-02-11 cs.CL cs.AI 62%

A Semi-Supervised Text Generation Framework Combining a Deep Transformer and a GAN

Shengquan Wang

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.AI

Comments 7 pages

2408.10189 2025-02-11 cs.LG cs.AI 62%

Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models

Aviv Bick, Kevin Y. Li, Eric P. Xing, J. Zico Kolter, Albert Gu

专题命中预训练与数据：language model(abstract)；分类 cs.AI、cs.LG

2502.04351 2025-02-10 cs.CL cs.AI 62%

NER4all or Context is All You Need: Using LLMs for low-effort, high-performance NER on historical texts. A humanities informed approach

Torsten Hiltmann, Martin Dröge, Nicole Dresselhaus, Till Grallert, Melanie Althage, Paul Bayer, Sophie Eckenstaler, Koray Mendi, Jascha Marijn Schmitz, Philipp Schneider, Wiebke Sczeponik, Anica Skibba

专题命中预训练与数据：prompting(abstract)；分类 cs.CL、cs.AI

2406.03857 2025-02-07 cs.LG cs.CL cs.CV 62%

MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognition

Stefan Gerd Fritsch, Cennet Oguz, Vitor Fortes Rey, Lala Ray, Maximilian Kiefer-Emmanouilidis, Paul Lukowicz

机构 * DFKI（德国人工智能研究中心）； RPTU Kaiserslautern-Landau（凯撒斯劳滕-兰道莱茵-普法尔茨大学）

专题命中预训练与数据：foundation model(abstract)；分类 cs.CL、cs.LG

2502.03128 2025-02-06 cs.SD cs.AI cs.LG eess.AS eess.SP 62%

Metis: A Foundation Speech Generation Model with Masked Generative Pre-training

Yuancheng Wang, Jiachen Zheng, Junan Zhang, Xueyao Zhang, Huan Liao, Zhizheng Wu

机构 * The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））

专题命中预训练与数据：foundation model(abstract)；分类 cs.AI、cs.LG

2202.08835 2025-01-17 cs.LG cs.AI cs.CV stat.ML 62%

General Cyclical Training of Neural Networks

Leslie N. Smith

专题命中预训练与数据：pretraining(abstract)；分类 cs.AI、cs.LG

Comments Position paper

Journal ref AAIML.2023.1157

2410.05450 2025-01-14 cs.CV cs.AI cs.LG 62%

AI-Driven Early Mental Health Screening: Analyzing Selfies of Pregnant Women

Gustavo A. Basílio, Thiago B. Pereira, Alessandro L. Koerich, Hermano Tavares, Ludmila Dias, Maria das Graças da S. Teixeira, Rafael T. Sousa, Wilian H. Hisatugu, Amanda S. Mota, Anilton S. Garcia, Marco Aurélio K. Galletta, Thiago M. Paixão

机构 * Federal Institute of Espírito Santo (IFES)（圣埃斯皮里图联邦学院）； École de Technologie Supérieure (ÉTS)（高等技术学院）； University of São Paulo Medical School (FMUSP)（圣保罗大学医学院）； Federal University of Espírito Santo (UFES)（圣埃斯皮里图联邦大学）； Federal University of Mato Grosso (UFMT)（马托格罗索联邦大学）

专题命中预训练与数据：language model(abstract)；分类 cs.AI、cs.LG

Comments This article has been accepted for publication in HEALTHINF25 at the 18th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2025)

2410.02780 2025-01-13 cs.CV cs.AI cs.LG 62%

Guess What I Think: Streamlined EEG-to-Image Generation with Latent Diffusion Models

Eleonora Lopez, Luigi Sigillo, Federica Colonnese, Massimo Panella, Danilo Comminiello

机构 * Sapienza University of Rome（罗马大学）

专题命中预训练与数据：pretraining(abstract)；分类 cs.AI、cs.LG

Comments Accepted at ICASSP 2025

2501.02156 2025-01-09 cs.LG cs.AI cs.PF 62%

The Race to Efficiency: A New Perspective on AI Scaling Laws

Chien-Ping Lu

机构 * Nimbyss

专题命中预训练与数据：prompting(abstract)；分类 cs.AI、cs.LG

Comments 21 pages, 3 figures. 2 tables, second draft

2501.04155 2025-01-09 cs.CV cs.CL cs.LG 62%

MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curation

Siddharth Joshi, Besmira Nushi, Vidhisha Balachandran, Varun Chandrasekaran, Vibhav Vineet, Neel Joshi, Baharan Mirzasoleiman

机构 * Microsoft Research（微软研究院）； UCLA（加利福尼亚大学洛杉矶分校）； UIUC（伊利诺伊大学厄巴纳-香槟分校）

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.LG

2501.04153 2025-01-09 cs.CL cs.LG 62%

Multilingual Open QA on the MIA Shared Task

Navya Yarrabelly, Saloni Mittal, Ketan Todi, Kimihiro Hasegawa

机构 * Language Technologies Institute（语言技术研究所）

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.LG

2501.03540 2025-01-08 cs.LG cs.AI 62%

Deep Learning within Tabular Data: Foundations, Challenges, Advances and Future Directions

Weijieying Ren, Tianxiang Zhao, Yuqing Huang, Vasant Honavar

机构 * The Pennsylvania State University（宾夕法尼亚州立大学）； University of Science and Technology of China（中国科学技术大学）

专题命中预训练与数据：foundation model(abstract)；分类 cs.AI、cs.LG

2412.10182 2025-01-07 cs.CV cs.AI cs.LG 62%

Multi-Head Encoding for Extreme Label Classification

Daojun Liang, Haixia Zhang, Dongfeng Yuan, Minggao Zhang

机构 * Shandong University（山东大学）； School of Information Science and Engineering, Shandong University（山东大学信息科学与工程学院）； School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）； School of Qilu Transportation, Shandong University（山东大学齐鲁交通学院）

专题命中预训练与数据：pretraining(abstract)；分类 cs.AI、cs.LG

Comments 20 pages, 12 figs, Published in IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2024

2501.01069 2025-01-03 cs.CL cs.LG 62%

BeliN: A Novel Corpus for Bengali Religious News Headline Generation using Contextual Feature Fusion

Md Osama, Ashim Dey, Kawsar Ahmed, Muhammad Ashad Kabir

机构 * Chittagong University of Engineering and Technology（吉大港工程技术大学）； Bangladesh Army University of Science and Technology (BAUST)（孟加拉国陆军科技大学）； Charles Sturt University（查尔斯斯特大学）

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.LG

Comments 28 pages, 4 figures, 11 tables

2411.17470 2025-01-03 cs.CV cs.AI cs.LG 62%

Towards Precise Scaling Laws for Video Diffusion Transformers

Yuanyang Yin, Yaqi Zhao, Mingwu Zheng, Ke Lin, Jiarong Ou, Rui Chen, Victor Shea-Jay Huang, Jiahao Wang, Xin Tao, Pengfei Wan, Di Zhang, Baoqun Yin, Wentao Zhang, Kun Gai

机构 * University of Science and Technology of China（中国科学技术大学）； Peking University（北京大学）； Kuaishou Technology（快手科技）

专题命中预训练与数据：language model(abstract)；分类 cs.AI、cs.LG

2412.18917 2024-12-30 cs.CV cs.AI cs.LG 62%

Open-Vocabulary Panoptic Segmentation Using BERT Pre-Training of Vision-Language Multiway Transformer Model

Yi-Chia Chen, Wei-Hua Li, Chu-Song Chen

机构 * National Taiwan University（台湾大学）

专题命中预训练与数据：foundation model(abstract)；分类 cs.AI、cs.LG

Comments ICIP 2024

2402.02262 2024-12-23 cs.CL cs.AI 62%

Data Quality Matters: Suicide Intention Detection on Social Media Posts Using RoBERTa-CNN

Emily Lin, Jian Sun, Hsingyu Chen, Mohammad H. Mahoor

机构 * University of Denver（丹佛大学）

专题命中预训练与数据：pretraining(abstract)；分类 cs.CL、cs.AI

Comments 4 pages, 1 figure, 4 tables

Journal ref 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 2024, pp. 1-5

2412.15821 2024-12-23 cs.CL cs.AI 62%

$π$-yalli: un nouveau corpus pour le nahuatl

Juan-Manuel Torres-Moreno, Juan-José Guzmán-Landa, Graham Ranger, Martha Lorena Avendaño Garrido, Miguel Figueroa-Saavedra, Ligia Quintana-Torres, Carlos-Emiliano González-Gallardo, Elvys Linhares Pontes, Patricia Velázquez Morales, Luis-Gil Moreno Jiménez

机构 * IEE, Universidad Veracruzana（韦拉克鲁斯大学IEE）； LIFAT, Université François Rabelais à Tours（图尔弗朗索瓦·拉伯雷大学LIFAT）； Trading Central Labs, Trading Central（Trading Central实验室,Trading Central）

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, in French language, 2 figures

2411.01929 2024-12-20 cs.LG cs.AI 62%

Exploring the Landscape for Generative Sequence Models for Specialized Data Synthesis

Mohammad Zbeeb, Mohammad Ghorayeb, Mariam Salman

机构 * American University of Beirut（贝鲁特美国大学）

专题命中预训练与数据：language model(abstract)；分类 cs.AI、cs.LG

Comments 25 pages, 7 figures, 3 tables, 1 algorithm. code @ https://github.com/Moe-Zbeeb/Exploring-the-landscape-for-generative-models-for-specialized-data-generation.git

Journal ref J Data Analytic Eng Decision Making, 1(2), 01-14 (2024)

2412.13924 2024-12-19 cs.CL cs.LG 62%

Language verY Rare for All

Ibrahim Merad, Amos Wolf, Ziad Mazzawi, Yannick Léo

机构 * Kaukana Ventures（考卡纳风险投资公司）； Emerton Data（埃默顿数据公司）

专题命中预训练与数据：LLM(abstract)；分类 cs.CL、cs.LG

2411.04434 2024-12-19 cs.LG cs.AI 62%

Scaling Laws for Pre-training Agents and World Models

Tim Pearce, Tabish Rashid, Dave Bignell, Raluca Georgescu, Sam Devlin, Katja Hofmann

机构 * Microsoft Research（微软研究院）

专题命中预训练与数据：language model(abstract)；分类 cs.AI、cs.LG

2407.00478 2024-12-18 cs.LG cs.AI 62%

Beyond Scaleup: Knowledge-aware Parsimony Learning from Deep Networks

Quanming Yao, Yongqi Zhang, Yaqing Wang, Nan Yin, James Kwok, Qiang Yang

机构 * Tsinghua University（清华大学）； Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））； Beijing Institute of Mathematical Sciences and Applications（北京数学科学与应用研究院）； Hong Kong University of Science and Technology（香港科技大学）

专题命中预训练与数据：foundation model(abstract)；分类 cs.AI、cs.LG

Comments Accepted to AI Magazine

2412.11691 2024-12-17 cs.CL cs.AI 62%

Multilingual and Explainable Text Detoxification with Parallel Corpora

Daryna Dementieva, Nikolay Babakov, Amit Ronen, Abinew Ali Ayele, Naquee Rizwan, Florian Schneider, Xintong Wang, Seid Muhie Yimam, Daniil Moskovskiy, Elisei Stakovskii, Eran Kaufman, Ashraf Elnagar, Animesh Mukherjee, Alexander Panchenko

机构 * Technical University of Munich（慕尼黑工业大学）； Universidade de Santiago de Compostela（圣地亚哥-德孔波斯特拉大学）； Shenkar College（申卡尔学院）； University of Hamburg（汉堡大学）； Bahir Dar University（巴达尔大学）； IIT Kharagpur（印度理工学院克勒格布尔分校）； Skoltech（斯科尔科沃科技学院）； AIRI（人工智能研究所）； University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）； University of Sharjah（沙迦大学）

专题命中预训练与数据：prompting(abstract)；分类 cs.CL、cs.AI

Comments COLING 2025, main conference, long

2412.11085 2024-12-17 cs.LG cs.AI 62%

GraphMoRE: Mitigating Topological Heterogeneity via Mixture of Riemannian Experts

Zihao Guo, Qingyun Sun, Haonan Yuan, Xingcheng Fu, Min Zhou, Yisen Gao, Jianxin Li

专题命中预训练与数据：foundation model(abstract)；分类 cs.AI、cs.LG

Comments Accepted by the Main Technical Track of the 39th Annual AAAI Conference on Artificial Intelligence (AAAI-2025)

2403.13804 2024-12-17 cs.CV cs.CL cs.LG 62%

Learning from Synthetic Data for Visual Grounding

Ruozhen He, Ziyan Yang, Paola Cascante-Bonilla, Alexander C. Berg, Vicente Ordonez

机构 * Rice University（莱斯大学）； University of Maryland（马里兰大学）； University of California, Irvine（加州大学欧文分校）

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.LG

Comments Project Page: https://catherine-r-he.github.io/SynGround/

2412.08680 2024-12-13 cs.CR cs.AI cs.HC cs.LG 62%

Distinguishing Scams and Fraud with Ensemble Learning

Isha Chadalavada, Tianhui Huang, Jessica Staddon

机构 * Northeastern University（东北大学）

专题命中预训练与数据：LLM(abstract)；分类 cs.AI、cs.LG

2411.06798 2024-12-12 q-bio.GN cs.AI cs.CL q-bio.QM 62%

LA4SR: illuminating the dark proteome with generative AI

David R. Nelson, Ashish Kumar Jaiswal, Noha Ismail, Alexandra Mystikou, Kourosh Salehi-Ashtiani

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.AI

1. 预训练与数据 12486 篇

Exploring RWKV for Sentence Embeddings: Layer-wise Analysis and Baseline Comparison for Semantic Similarity

Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling

Syntriever: How to Train Your Retriever with Synthetic Data from LLMs

A Semi-Supervised Text Generation Framework Combining a Deep Transformer and a GAN

Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models

NER4all or Context is All You Need: Using LLMs for low-effort, high-performance NER on historical texts. A humanities informed approach

MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognition

Metis: A Foundation Speech Generation Model with Masked Generative Pre-training

General Cyclical Training of Neural Networks

AI-Driven Early Mental Health Screening: Analyzing Selfies of Pregnant Women

Guess What I Think: Streamlined EEG-to-Image Generation with Latent Diffusion Models

The Race to Efficiency: A New Perspective on AI Scaling Laws

MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curation

Multilingual Open QA on the MIA Shared Task

Deep Learning within Tabular Data: Foundations, Challenges, Advances and Future Directions

Multi-Head Encoding for Extreme Label Classification

BeliN: A Novel Corpus for Bengali Religious News Headline Generation using Contextual Feature Fusion

Towards Precise Scaling Laws for Video Diffusion Transformers

Open-Vocabulary Panoptic Segmentation Using BERT Pre-Training of Vision-Language Multiway Transformer Model

Data Quality Matters: Suicide Intention Detection on Social Media Posts Using RoBERTa-CNN

$π$-yalli: un nouveau corpus pour le nahuatl

Exploring the Landscape for Generative Sequence Models for Specialized Data Synthesis

Language verY Rare for All

Scaling Laws for Pre-training Agents and World Models

Beyond Scaleup: Knowledge-aware Parsimony Learning from Deep Networks

Multilingual and Explainable Text Detoxification with Parallel Corpora

GraphMoRE: Mitigating Topological Heterogeneity via Mixture of Riemannian Experts

Learning from Synthetic Data for Visual Grounding

Distinguishing Scams and Fraud with Ensemble Learning

LA4SR: illuminating the dark proteome with generative AI