语言大模型 / LLM

Training Bilingual LMs with Data Constraints in the Targeted Language

Skyler Seto, Maartje ter Hoeve, Richard He Bai, Natalie Schluter, David Grangier

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 26 pages, 22 figures, 15 tables

Time-Reversal Provides Unsupervised Feedback to LLMs

Yerram Varun, Rahul Madhavan, Sravanti Addepalli, Arun Suggala, Karthikeyan Shanmugam, Prateek Jain

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments Accepted as a spotlight in NeurIPS 2024

Journal ref The Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS), 2024

BreezyVoice: Adapting TTS for Taiwanese Mandarin with Enhanced Polyphone Disambiguation -- Challenges and Insights

Chan-Jan Hsu, Yi-Cheng Lin, Chia-Chun Lin, Wei-Chih Chen, Ho Lam Chung, Chen-An Li, Yi-Chang Chen, Chien-Yu Yu, Ming-Ji Lee, Chien-Cheng Chen, Ru-Heng Huang, Hung-yi Lee, Da-Shan Shiu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

FactCG: Enhancing Fact Checkers with Graph-Based Multi-Hop Data

Deren Lei, Yaxi Li, Siyao Li, Mengya Hu, Rui Xu, Ken Archer, Mingyu Wang, Emily Ching, Alex Deng

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments NAACL 2025

Explaining Categorical Feature Interactions Using Graph Covariance and LLMs

Cencheng Shen, Darren Edge, Jonathan Larson, Carey E. Priebe

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 18 pages main + 6 pages appendix

T3: A Novel Zero-shot Transfer Learning Framework Iteratively Training on an Assistant Task for a Target Task

Xindi Tong, Yujin Zhu, Shijian Fan, Liang Xu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments

Hongjin Su, Ruoxi Sun, Jinsung Yoon, Pengcheng Yin, Tao Yu, Sercan Ö. Arık

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Bridging Language Barriers in Healthcare: A Study on Arabic LLMs

Nada Saadi, Tathagata Raha, Clément Christophe, Marco AF Pimentel, Ronnie Rajan, Praveen K Kanithi

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limit

Oleg Filatov, Jan Ebert, Jiangtao Wang, Stefan Kesselheim

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Rho-1: Not All Tokens Are What You Need

Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Chen Lin, Yujiu Yang, Jian Jiao, Nan Duan, Weizhu Chen

专题命中 预训练与数据 ：language model(abstract)；SLM(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments First two authors equal contribution

Rethinking Data Synthesis: A Teacher Model Training Recipe with Interpretation

Yifang Chen, David Zhu, Simon Du, Kevin Jamieson, Yang Liu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

A Simple Model of Inference Scaling Laws

Noam Levi

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.AI、cs.LG

Comments 12 pages, 7 figures

Are LLMs good pragmatic speakers?

Mingyue Jian, N. Siddharth

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Scaling Laws for Precision

Tanishq Kumar, Zachary Ankner, Benjamin F. Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Ré, Aditi Raghunathan

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；post-training(abstract)；分类 cs.CL、cs.LG

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens

Xu Ouyang, Tao Ge, Thomas Hartvigsen, Zhisong Zhang, Haitao Mi, Dong Yu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Work in Progress

Predicting Emergent Capabilities by Finetuning

Charlie Snell, Eric Wallace, Dan Klein, Sergey Levine

专题命中 预训练与数据 ：LLM(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Alopex: A Computational Framework for Enabling On-Device Function Calls with LLMs

Yide Ran, Zhaozhuo Xu, Yuhang Yao, Zijian Hu, Shanshan Han, Han Jin, Alay Dilipbhai Shah, Jipeng Zhang, Dimitris Stripelis, Tong Zhang, Salman Avestimehr, Chaoyang He

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Bottom-Up and Top-Down Analysis of Values, Agendas, and Observations in Corpora and LLMs

Scott E. Friedman, Noam Benkler, Drisana Mosaphir, Jeffrey Rye, Sonja M. Schmer-Galunder, Micah Goldwater, Matthew McLure, Ruta Wheelock, Jeremy Gottlieb, Robert P. Goldman, Christopher Miller

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Bias in the Mirror: Are LLMs opinions robust to their own adversarial attacks ?

Virgile Rennard, Christos Xypolopoulos, Michalis Vazirgiannis

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Structured prompt interrogation and recursive extraction of semantics (SPIRES): A method for populating knowledge bases using zero-shot learning

J. Harry Caufield, Harshad Hegde, Vincent Emonet, Nomi L. Harris, Marcin P. Joachimiak, Nicolas Matentzoglu, HyeongSik Kim, Sierra A. T. Moxon, Justin T. Reese, Melissa A. Haendel, Peter N. Robinson, Christopher J. Mungall

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Updated 2023-12-22

Journal ref Bioinformatics, 2024 Mar 4;40(3)

CurateGPT: A flexible language-model assisted biocuration tool

Harry Caufield, Carlo Kroll, Shawn T O'Neil, Justin T Reese, Marcin P Joachimiak, Harshad Hegde, Nomi L Harris, Madan Krishnamurthy, James A McLaughlin, Damian Smedley, Melissa A Haendel, Peter N Robinson, Christopher J Mungall

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies

Chaofan Tao, Qian Liu, Longxu Dou, Niklas Muennighoff, Zhongwei Wan, Ping Luo, Min Lin, Ngai Wong

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024

Waterfall: Framework for Robust and Scalable Text Watermarking and Provenance for LLMs

Gregory Kang Ruey Lau, Xinyuan Niu, Hieu Dao, Jiangwei Chen, Chuan-Sheng Foo, Bryan Kian Hsiang Low

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2024 Main Conference

CodeRosetta: Pushing the Boundaries of Unsupervised Code Translation for Parallel Programming

Ali TehraniJamsaz, Arijit Bhattacharjee, Le Chen, Nesreen K. Ahmed, Amir Yazdanbakhsh, Ali Jannesari

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

End-To-End Causal Effect Estimation from Unstructured Natural Language Data

Nikita Dhawan, Leonardo Cotta, Karen Ullrich, Rahul G. Krishnan, Chris J. Maddison

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2024

DALD: Improving Logits-based Detector without Logits from Black-box LLMs

Cong Zeng, Shengkun Tang, Xianjun Yang, Yuanzhou Chen, Yiyou Sun, zhiqiang xu, Yao Li, Haifeng Chen, Wei Cheng, Dongkuan Xu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification

Yunzhen Feng, Elvis Dohmatob, Pu Yang, Francois Charton, Julia Kempe

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Detecting AI-Generated Texts in Cross-Domains

You Zhou, Jie Wang

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Journal ref DocEng '24: Proceedings of the ACM Symposium on Document Engineering 2024

MindFormer: Semantic Alignment of Multi-Subject fMRI for Brain Decoding

Inhwa Han, Jaayeon Lee, Jong Chul Ye

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Learning Semantic Structure through First-Order-Logic Translation

Akshay Chaturvedi, Nicholas Asher

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.LG

Comments EMNLP 2024 Findings