语言大模型 / LLM

Bridging Language Barriers in Healthcare: A Study on Arabic LLMs

Nada Saadi, Tathagata Raha, Clément Christophe, Marco AF Pimentel, Ronnie Rajan, Praveen K Kanithi

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limit

Oleg Filatov, Jan Ebert, Jiangtao Wang, Stefan Kesselheim

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Rho-1: Not All Tokens Are What You Need

Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Chen Lin, Yujiu Yang, Jian Jiao, Nan Duan, Weizhu Chen

专题命中 预训练与数据 ：language model(abstract)；SLM(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments First two authors equal contribution

Rethinking Data Synthesis: A Teacher Model Training Recipe with Interpretation

Yifang Chen, David Zhu, Simon Du, Kevin Jamieson, Yang Liu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

A Simple Model of Inference Scaling Laws

Noam Levi

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.AI、cs.LG

Comments 12 pages, 7 figures

Are LLMs good pragmatic speakers?

Mingyue Jian, N. Siddharth

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Scaling Laws for Precision

Tanishq Kumar, Zachary Ankner, Benjamin F. Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Ré, Aditi Raghunathan

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；post-training(abstract)；分类 cs.CL、cs.LG

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens

Xu Ouyang, Tao Ge, Thomas Hartvigsen, Zhisong Zhang, Haitao Mi, Dong Yu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Work in Progress

Predicting Emergent Capabilities by Finetuning

Charlie Snell, Eric Wallace, Dan Klein, Sergey Levine

专题命中 预训练与数据 ：LLM(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Alopex: A Computational Framework for Enabling On-Device Function Calls with LLMs

Yide Ran, Zhaozhuo Xu, Yuhang Yao, Zijian Hu, Shanshan Han, Han Jin, Alay Dilipbhai Shah, Jipeng Zhang, Dimitris Stripelis, Tong Zhang, Salman Avestimehr, Chaoyang He

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Bottom-Up and Top-Down Analysis of Values, Agendas, and Observations in Corpora and LLMs

Scott E. Friedman, Noam Benkler, Drisana Mosaphir, Jeffrey Rye, Sonja M. Schmer-Galunder, Micah Goldwater, Matthew McLure, Ruta Wheelock, Jeremy Gottlieb, Robert P. Goldman, Christopher Miller

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Bias in the Mirror: Are LLMs opinions robust to their own adversarial attacks ?

Virgile Rennard, Christos Xypolopoulos, Michalis Vazirgiannis

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Structured prompt interrogation and recursive extraction of semantics (SPIRES): A method for populating knowledge bases using zero-shot learning

J. Harry Caufield, Harshad Hegde, Vincent Emonet, Nomi L. Harris, Marcin P. Joachimiak, Nicolas Matentzoglu, HyeongSik Kim, Sierra A. T. Moxon, Justin T. Reese, Melissa A. Haendel, Peter N. Robinson, Christopher J. Mungall

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Updated 2023-12-22

Journal ref Bioinformatics, 2024 Mar 4;40(3)

CurateGPT: A flexible language-model assisted biocuration tool

Harry Caufield, Carlo Kroll, Shawn T O'Neil, Justin T Reese, Marcin P Joachimiak, Harshad Hegde, Nomi L Harris, Madan Krishnamurthy, James A McLaughlin, Damian Smedley, Melissa A Haendel, Peter N Robinson, Christopher J Mungall

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies

Chaofan Tao, Qian Liu, Longxu Dou, Niklas Muennighoff, Zhongwei Wan, Ping Luo, Min Lin, Ngai Wong

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024

Waterfall: Framework for Robust and Scalable Text Watermarking and Provenance for LLMs

Gregory Kang Ruey Lau, Xinyuan Niu, Hieu Dao, Jiangwei Chen, Chuan-Sheng Foo, Bryan Kian Hsiang Low

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2024 Main Conference

CodeRosetta: Pushing the Boundaries of Unsupervised Code Translation for Parallel Programming

Ali TehraniJamsaz, Arijit Bhattacharjee, Le Chen, Nesreen K. Ahmed, Amir Yazdanbakhsh, Ali Jannesari

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

End-To-End Causal Effect Estimation from Unstructured Natural Language Data

Nikita Dhawan, Leonardo Cotta, Karen Ullrich, Rahul G. Krishnan, Chris J. Maddison

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2024

DALD: Improving Logits-based Detector without Logits from Black-box LLMs

Cong Zeng, Shengkun Tang, Xianjun Yang, Yuanzhou Chen, Yiyou Sun, zhiqiang xu, Yao Li, Haifeng Chen, Wei Cheng, Dongkuan Xu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification

Yunzhen Feng, Elvis Dohmatob, Pu Yang, Francois Charton, Julia Kempe

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Detecting AI-Generated Texts in Cross-Domains

You Zhou, Jie Wang

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Journal ref DocEng '24: Proceedings of the ACM Symposium on Document Engineering 2024

MindFormer: Semantic Alignment of Multi-Subject fMRI for Brain Decoding

Inhwa Han, Jaayeon Lee, Jong Chul Ye

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Learning Semantic Structure through First-Order-Logic Translation

Akshay Chaturvedi, Nicholas Asher

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.LG

Comments EMNLP 2024 Findings

Dynamic Gradient Alignment for Online Data Mixing

Simin Fan, David Grangier, Pierre Ablin

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Extracting Memorized Training Data via Decomposition

Ellen Su, Anu Vellore, Amy Chang, Raffaele Mura, Blaine Nelson, Paul Kassianik, Amin Karbasi

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

MiniConGTS: A Near Ultimate Minimalist Contrastive Grid Tagging Scheme for Aspect Sentiment Triplet Extraction

Qiao Sun, Liujia Yang, Minghao Ma, Nanyang Ye, Qinying Gu

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments arXiv admin note: text overlap with arXiv:2403.07342

GEM-RAG: Graphical Eigen Memories For Retrieval Augmented Generation

Brendan Hogan Rappazzo, Yingheng Wang, Aaron Ferber, Carla Gomes

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 8 pages

Column Vocabulary Association (CVA): semantic interpretation of dataless tables

Margherita Martorana, Xueli Pan, Benno Kruit, Tobias Kuhn, Jacco van Ossenbruggen

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

HIRO: Hierarchical Information Retrieval Optimization

Krish Goel, Mahek Chandak

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Diagnosing and Remedying Knowledge Deficiencies in LLMs via Label-free Curricular Meaningful Learning

Kai Xiong, Xiao Ding, Li Du, Jiahao Ying, Ting Liu, Bing Qin, Yixin Cao

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Under Review