语言大模型 / LLM - arXivDaily 专题

2501.02044 2025-01-07 cs.CL cs.AI 73%

Advancing Pancreatic Cancer Prediction with a Next Visit Token Prediction Head on top of Med-BERT

Jianping He, Laila Rasmy, Degui Zhi, Cui Tao

专题命中预训练与数据：foundation model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

2412.20024 2025-01-07 cs.AI cs.CL 73%

BaiJia: A Large-Scale Role-Playing Agent Corpus of Chinese Historical Characters

Ting Bai, Jiazheng Kang, Jiayang Fan

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2306.17181 2025-01-07 cs.CL cs.LG 73%

Unsupervised Text Embedding Space Generation Using Generative Adversarial Networks for Text Synthesis

Jun-Min Lee, Tae-Bin Ha

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments NEJLT accpeted

2412.16201 2024-12-24 cs.RO cs.AI cs.LG cs.SY eess.SY 73%

CLIP-RLDrive: Human-Aligned Autonomous Driving via CLIP-Based Reward Shaping in Reinforcement Learning

Erfan Doroudian, Hamid Taghavifar

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

2412.08905 2024-12-13 cs.CL cs.AI 73%

Phi-4 Technical Report

Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, Yi Zhang

专题命中预训练与数据：language model(abstract)；post-training(abstract)；分类 cs.CL、cs.AI

2412.08737 2024-12-13 cs.CV cs.AI cs.CL 73%

Euclid: Supercharging Multimodal LLMs with Synthetic High-Fidelity Visual Descriptions

Jiarui Zhang, Ollie Liu, Tianyu Yu, Jinyi Hu, Willie Neiswanger

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 33 pages, 22 figures, 5 tables, 7 algorithms

2406.18814 2024-12-12 stat.ML cs.AI cs.LG stat.ME 73%

Length Optimization in Conformal Prediction

Shayan Kiyani, George Pappas, Hamed Hassani

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2412.06176 2024-12-10 cs.LG cs.AI 73%

AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement

Pranjal Aggarwal, Bryan Parno, Sean Welleck

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2410.05192 2024-12-04 cs.LG cs.CL stat.ML 73%

Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective

Kaiyue Wen, Zhiyuan Li, Jason Wang, David Hall, Percy Liang, Tengyu Ma

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 45 pages,13 figures

2407.16607 2024-12-03 cs.CL cs.LG 73%

Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?

Jonathan Hayase, Alisa Liu, Yejin Choi, Sewoong Oh, Noah A. Smith

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments NeurIPS camera-ready, code at https://github.com/alisawuffles/tokenizer-attack

2411.09612 2024-11-15 cs.CL cs.LG 73%

The Moral Foundations Weibo Corpus

Renjie Cao, Miaoyan Hu, Jiahan Wei, Baha Ihnaini

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2406.00131 2024-11-12 cs.LG cs.CL stat.ML 73%

From Unstructured Data to In-Context Learning: Exploring What Tasks Can Be Learned and When

Kevin Christian Wibisono, Yixin Wang

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 39 pages

2411.02393 2024-11-05 cs.CV cs.AI cs.LG cs.RO 73%

Adaptive Length Image Tokenization via Recurrent Allocation

Shivam Duggal, Phillip Isola, Antonio Torralba, William T. Freeman

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Code at: https://github.com/ShivamDuggal4/adaptive-length-tokenizer

2410.17744 2024-11-05 cs.LG cs.AI 73%

Learning Versatile Skills with Curriculum Masking

Yao Tang, Zhihui Xie, Zichuan Lin, Deheng Ye, Shuai Li

专题命中预训练与数据：pretraining(abstract)；prompting(abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2024 poster, 21 pages, 8 figures

2410.23438 2024-11-01 cs.LG cs.CL 73%

Learning and Transferring Sparse Contextual Bigrams with Linear Transformers

Yunwei Ren, Zixuan Wang, Jason D. Lee

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

2409.19759 2024-10-31 cs.CL cs.LG 73%

Balancing Cost and Effectiveness of Synthetic Data Generation Strategies for LLMs

Yung-Chieh Chan, George Pu, Apaar Shanker, Parth Suresh, Penn Jenks, John Heyer, Sam Denton

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments NeurIPS '24 Workshop on Fine-Tuning in Modern Machine Learning: Principles and Scalability

2309.06703 2024-10-29 cs.CV cs.CL cs.HC cs.LG 73%

VLSlice: Interactive Vision-and-Language Slice Discovery

Eric Slyman, Minsuk Kahng, Stefan Lee

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments Conference paper at ICCV 2023. 17 pages, 11 figures. https://ericslyman.com/vlslice/

Journal ref 2023 IEEE/CVF International Conference on Computer Vision (ICCV)

2402.18153 2024-10-28 cs.LG cs.AI 73%

Diffusion-Based Neural Network Weights Generation

Bedionita Soro, Bruno Andreis, Hayeon Lee, Wonyong Jeong, Song Chong, Frank Hutter, Sung Ju Hwang

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 32 pages

2404.07242 2024-10-22 cs.CR cs.AI cs.CL 73%

Sandwich attack: Multi-language Mixture Adaptive Attack on LLMs

Bibek Upadhayay, Vahid Behzadan

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2410.10801 2024-10-15 cs.CL cs.LG 73%

Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning

Aakanksha, Arash Ahmadian, Seraphina Goldfarb-Tarrant, Beyza Ermis, Marzieh Fadaee, Sara Hooker

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2410.08942 2024-10-14 cs.LG cs.AI math.ST stat.TH 73%

Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory

Aymane El Firdoussi, Mohamed El Amine Seddik, Soufiane Hayou, Reda Alami, Ahmed Alzubaidi, Hakim Hacid

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2410.06311 2024-10-10 cs.IR cs.AI cs.LG 73%

A Comparative Study of Hybrid Models in Health Misinformation Text Classification

Mkululi Sikosana, Oluwaseun Ajao, Sean Maudsley-Barton

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments 8 pages, 4 tables presented at the OASIS workshop of the ACM Hypertext and Social Media Conference 2024

Journal ref In Proceedings of the 4th International Workshop on Open Challenges in Online Social Networks (pp. 18-25) 2024

2410.04002 2024-10-08 cs.CL cs.AI 73%

Take It Easy: Label-Adaptive Self-Rationalization for Fact Verification and Explanation Generation

Jing Yang, Anderson Rocha

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Paper accepted in the 16th IEEE INTERNATIONAL WORKSHOP ON INFORMATION FORENSICS AND SECURITY (WIFS) 2024

2408.16221 2024-10-07 eess.AS cs.AI cs.CL cs.SD 73%

SSDM: Scalable Speech Dysfluency Modeling

Jiachen Lian, Xuanru Zhou, Zoe Ezzes, Jet Vonk, Brittany Morin, David Baquirin, Zachary Mille, Maria Luisa Gorno Tempini, Gopala Krishna Anumanchipalli

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 2024 NeurIPS

2311.08644 2024-10-07 cs.LG cs.AI cs.HC 73%

Wrapper Boxes: Faithful Attribution of Model Predictions to Training Data

Yiheng Su, Junyi Jessy Li, Matthew Lease

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Journal ref The seventh edition of BlackboxNLP Workshop at EMNLP 2024

2409.20054 2024-10-01 cs.CL cs.AI 73%

Evaluating and explaining training strategies for zero-shot cross-lingual news sentiment analysis

Luka Andrenšek, Boshko Koloski, Andraž Pelicon, Nada Lavrač, Senja Pollak, Matthew Purver

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments The first two authors share equal contribution

2409.16341 2024-09-27 cs.LG cs.CL cs.SE 73%

Quality Matters: Evaluating Synthetic Data for Tool-Using LLMs

Shadi Iskander, Nachshon Cohen, Zohar Karnin, Ori Shapira, Sofia Tolmach

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2409.15794 2024-09-25 cs.LG cs.AI 73%

Towards Universal Large-Scale Foundational Model for Natural Gas Demand Forecasting

Xinxing Zhou, Jiaqi Ye, Shubao Zhao, Ming Jin, Zhaoxiang Hou, Chengyi Yang, Zengxiang Li, Yanlong Wen, Xiaojie Yuan

专题命中预训练与数据：foundation model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

2409.15461 2024-09-25 cs.AI cs.CL 73%

RAM2C: A Liberal Arts Educational Chatbot based on Retrieval-augmented Multi-role Multi-expert Collaboration

Haoyu Huang, Tong Niu, Rui Yang, Luping Shi

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2409.15051 2024-09-24 cs.CL cs.AI 73%

Scaling Laws of Decoder-Only Models on the Multilingual Machine Translation Task

Gaëtan Caillaut, Raheel Qader, Mariam Nakhlé, Jingshu Liu, Jean-Gabriel Barthélemy

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

1. 预训练与数据 12460 篇

Advancing Pancreatic Cancer Prediction with a Next Visit Token Prediction Head on top of Med-BERT

BaiJia: A Large-Scale Role-Playing Agent Corpus of Chinese Historical Characters

Unsupervised Text Embedding Space Generation Using Generative Adversarial Networks for Text Synthesis

CLIP-RLDrive: Human-Aligned Autonomous Driving via CLIP-Based Reward Shaping in Reinforcement Learning

Phi-4 Technical Report

Euclid: Supercharging Multimodal LLMs with Synthetic High-Fidelity Visual Descriptions

Length Optimization in Conformal Prediction

AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement

Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective

Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?

The Moral Foundations Weibo Corpus

From Unstructured Data to In-Context Learning: Exploring What Tasks Can Be Learned and When

Adaptive Length Image Tokenization via Recurrent Allocation

Learning Versatile Skills with Curriculum Masking

Learning and Transferring Sparse Contextual Bigrams with Linear Transformers

Balancing Cost and Effectiveness of Synthetic Data Generation Strategies for LLMs

VLSlice: Interactive Vision-and-Language Slice Discovery

Diffusion-Based Neural Network Weights Generation

Sandwich attack: Multi-language Mixture Adaptive Attack on LLMs

Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning

Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory

A Comparative Study of Hybrid Models in Health Misinformation Text Classification

Take It Easy: Label-Adaptive Self-Rationalization for Fact Verification and Explanation Generation

SSDM: Scalable Speech Dysfluency Modeling

Wrapper Boxes: Faithful Attribution of Model Predictions to Training Data

Evaluating and explaining training strategies for zero-shot cross-lingual news sentiment analysis

Quality Matters: Evaluating Synthetic Data for Tool-Using LLMs

Towards Universal Large-Scale Foundational Model for Natural Gas Demand Forecasting

RAM2C: A Liberal Arts Educational Chatbot based on Retrieval-augmented Multi-role Multi-expert Collaboration

Scaling Laws of Decoder-Only Models on the Multilingual Machine Translation Task