语言大模型 / LLM - arXivDaily 专题

2411.04704 2024-11-08 cs.SE 85%

Distinguishing LLM-generated from Human-written Code by Contrastive Learning

Xiaodan Xu, Chao Ni, Xinrong Guo, Shaoxuan Liu, Xiaoya Wang, Kui Liu, Xiaohu Yang

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments 30 pages, 6 figures, Accepted by TOSEM'24

2410.24034 2024-11-01 cs.CV 85%

Handwriting Recognition in Historical Documents with Multimodal LLM

Lucian Li

专题命中预训练与数据：LLM(title,abstract)；language model(abstract)；prompting(abstract)

2410.23676 2024-11-01 cs.CV 85%

Web-Scale Visual Entity Recognition: An LLM-Driven Data Approach

Mathilde Caron, Alireza Fathi, Cordelia Schmid, Ahmet Iscen

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments NeurIPS 2024

2312.17120 2024-10-30 cs.CL cs.AI cs.LG 85%

MathPile: A Billion-Token-Scale Pretraining Corpus for Math

Zengzhi Wang, Xuefeng Li, Rui Xia, Pengfei Liu

专题命中预训练与数据：pretraining(title)；language model(abstract)；foundation model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 43 pages. Accepted by NeurIPS 2024. https://github.com/GAIR-NLP/MathPile/

2410.09687 2024-10-15 cs.LG cs.AI cs.CL 85%

MoIN: Mixture of Introvert Experts to Upcycle an LLM

Ajinkya Tejankar, KL Navaneet, Ujjawal Panchal, Kossar Pourahmadi, Hamed Pirsiavash

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2408.03617 2024-10-10 cs.CL cs.AI cs.LG 85%

Is Child-Directed Speech Effective Training Data for Language Models?

Steven Y. Feng, Noah D. Goodman, Michael C. Frank

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments EMNLP 2024. Code and data at https://github.com/styfeng/TinyDialogues

2409.07431 2024-10-04 cs.LG cs.AI cs.CL stat.ML 85%

Synthetic continued pretraining

Zitong Yang, Neil Band, Shuangping Li, Emmanuel Candès, Tatsunori Hashimoto

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Updated organization of experimental results and methods introduction. Released the dataset and model weights artifact

2405.10938 2024-10-03 cs.LG cs.AI cs.CL stat.ML 85%

Observational Scaling Laws and the Predictability of Language Model Performance

Yangjun Ruan, Chris J. Maddison, Tatsunori Hashimoto

专题命中预训练与数据：language model(title,abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at NeurIPS 2024 as a spotlight

2406.11176 2024-09-26 cs.CL cs.AI cs.LG 85%

Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinement

Weimin Xiong, Yifan Song, Xiutian Zhao, Wenhao Wu, Xun Wang, Ke Wang, Cheng Li, Wei Peng, Sujian Li

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to EMNLP 2024 (Main Conference)

2409.15678 2024-09-25 q-bio.QM q-bio.GN 85%

Objectively Evaluating the Reliability of Cell Type Annotation Using LLM-Based Strategies

Wenjin Ye, Yuanchen Ma, Junkai Xiang, Hongjie Liang, Tao Wang, Qiuling Xiang, Andy Peng Xiang, Wu Song, Weiqiang Li, Weijun Huang

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

2308.04029 2024-09-17 cs.RO 85%

ChatSim: Underwater Simulation with Natural Language Prompting

Aadi Palnitkar, Rashmi Kapu, Xiaomin Lin, Cheng Liu, Nare Karapetyan, Yiannis Aloimonos

专题命中预训练与数据：prompting(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

Journal ref OCEANS 2023 - MTS/IEEE U.S. Gulf Coast

2408.07091 2024-08-22 cs.LG cs.AI cs.CL 85%

Node Level Graph Autoencoder: Unified Pretraining for Textual Graph Learning

Wenbin Hu, Huihao Jing, Qi Hu, Haoran Li, Yangqiu Song

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2407.20371 2024-08-22 cs.CY cs.AI cs.CL cs.LG 85%

Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval

Kyra Wilson, Aylin Caliskan

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments To be published in Proceedings of the 2024 AAAI/ACM Conference on AI, Ethics, and Society; code available at https://github.com/kyrawilson/Resume-Screening-Bias

2308.07922 2024-08-20 cs.CL cs.AI cs.LG 85%

RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models

Jie Huang, Wei Ping, Peng Xu, Mohammad Shoeybi, Kevin Chen-Chuan Chang, Bryan Catanzaro

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments COLM 2024

2407.01490 2024-07-22 cs.CL cs.AI cs.LG 85%

LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectives

Luísa Shimabucoro, Sebastian Ruder, Julia Kreutzer, Marzieh Fadaee, Sara Hooker

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2407.12854 2024-07-19 cs.CL cs.AI cs.IR cs.LG 85%

Scaling Retrieval-Based Language Models with a Trillion-Token Datastore

Rulin Shao, Jacqueline He, Akari Asai, Weijia Shi, Tim Dettmers, Sewon Min, Luke Zettlemoyer, Pang Wei Koh

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

2406.13972 2024-07-09 cs.SE 85%

CREF: An LLM-based Conversational Software Repair Framework for Programming Tutors

Boyang Yang, Haoye Tian, Weiguo Pian, Haoran Yu, Haitao Wang, Jacques Klein, Tegawendé F. Bissyandé, Shunfu Jin

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)

2407.04489 2024-07-08 cs.CV 85%

Dude: Dual Distribution-Aware Context Prompt Learning For Large Vision-Language Model

Duy M. H. Nguyen, An T. Le, Trung Q. Nguyen, Nghiem T. Diep, Tai Nguyen, Duy Duong-Tran, Jan Peters, Li Shen, Mathias Niepert, Daniel Sonntag

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)

Comments Version 1

2406.12114 2024-06-19 cs.CL cs.AI cs.LG 85%

Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation

Hamidreza Rouzegar, Masoud Makrehchi

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Publisher: Association for Computational Linguistics URL: https://aclanthology.org/2024.law-1.10

Journal ref Proceedings of The 18th Linguistic Annotation Workshop (LAW-XVIII), 2024, pp. 98-111

2402.11485 2024-06-07 cs.CL cs.AI cs.LG 85%

LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation

Ikuya Yamada, Ryokan Ri

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL Findings 2024

2211.04325 2024-06-06 cs.LG cs.AI cs.CL cs.CV cs.CY 85%

Will we run out of data? Limits of LLM scaling based on human-generated data

Pablo Villalobos, Anson Ho, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, Marius Hobbhahn

专题命中预训练与数据：LLM(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2405.15728 2024-05-27 cs.CV 85%

Disease-informed Adaptation of Vision-Language Models

Jiajin Zhang, Ge Wang, Mannudeep K. Kalra, Pingkun Yan

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；prompting(abstract)

Comments Early Accepted by MICCAI 2024

2307.05222 2024-05-09 cs.CV 85%

Emu: Generative Pretraining in Multimodality

Quan Sun, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, Yueze Wang, Hongcheng Gao, Jingjing Liu, Tiejun Huang, Xinlong Wang

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)；instruction tuning(abstract)

Comments Accepted to ICLR 2024. Code and Models: https://github.com/baaivision/Emu

2310.02226 2024-04-23 cs.CL cs.AI cs.LG 85%

Think before you speak: Training Language Models With Pause Tokens

Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, Vaishnavh Nagarajan

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published at ICLR 2024

2404.02145 2024-04-18 cs.CV 85%

Iterated Learning Improves Compositionality in Large Vision-Language Models

Chenhao Zheng, Jieyu Zhang, Aniruddha Kembhavi, Ranjay Krishna

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；language agent(abstract)

Comments CVPR 2024

2404.07612 2024-04-12 cs.CY 85%

Measuring Geographic Diversity of Foundation Models with a Natural Language--based Geo-guessing Experiment on GPT-4

Zilong Liu, Krzysztof Janowicz, Kitty Currier, Meilin Shi

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)

Comments Short paper accepted by AGILE 2024 conference (https://agile-gi.eu/conference-2024)

2403.19063 2024-03-29 cs.IR 85%

Instruction-based Hypergraph Pretraining

Mingdai Yang, Zhiwei Liu, Liangwei Yang, Xiaolong Liu, Chen Wang, Hao Peng, Philip S. Yu

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；prompting(abstract)

Comments Accepted by SIGIR'24

2312.01629 2024-03-28 cs.CV 85%

CLAMP: Contrastive LAnguage Model Prompt-tuning

Piotr Teterwak, Ximeng Sun, Bryan A. Plummer, Kate Saenko, Ser-Nam Lim

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)

2306.00618 2024-03-22 cs.CL cs.AI cs.LG 85%

Effective Structured Prompting by Meta-Learning and Representative Verbalizer

Weisen Jiang, Yu Zhang, James T. Kwok

专题命中预训练与数据：prompting(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ICML 2023

2403.09629 2024-03-19 cs.CL cs.AI cs.LG 85%

Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking

Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, Noah D. Goodman

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

1. 预训练与数据 12429 篇

Distinguishing LLM-generated from Human-written Code by Contrastive Learning

Handwriting Recognition in Historical Documents with Multimodal LLM

Web-Scale Visual Entity Recognition: An LLM-Driven Data Approach

MathPile: A Billion-Token-Scale Pretraining Corpus for Math

MoIN: Mixture of Introvert Experts to Upcycle an LLM

Is Child-Directed Speech Effective Training Data for Language Models?

Synthetic continued pretraining

Observational Scaling Laws and the Predictability of Language Model Performance

Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinement

Objectively Evaluating the Reliability of Cell Type Annotation Using LLM-Based Strategies

ChatSim: Underwater Simulation with Natural Language Prompting

Node Level Graph Autoencoder: Unified Pretraining for Textual Graph Learning

Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval

RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models

LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectives

Scaling Retrieval-Based Language Models with a Trillion-Token Datastore

CREF: An LLM-based Conversational Software Repair Framework for Programming Tutors

Dude: Dual Distribution-Aware Context Prompt Learning For Large Vision-Language Model

Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation

LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation

Will we run out of data? Limits of LLM scaling based on human-generated data

Disease-informed Adaptation of Vision-Language Models

Emu: Generative Pretraining in Multimodality

Think before you speak: Training Language Models With Pause Tokens

Iterated Learning Improves Compositionality in Large Vision-Language Models

Measuring Geographic Diversity of Foundation Models with a Natural Language--based Geo-guessing Experiment on GPT-4

Instruction-based Hypergraph Pretraining

CLAMP: Contrastive LAnguage Model Prompt-tuning

Effective Structured Prompting by Meta-Learning and Representative Verbalizer

Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking