语言大模型 / LLM

Retentive or Forgetful? Diving into the Knowledge Memorizing Mechanism of Language Models

Boxi Cao, Qiaoyu Tang, Hongyu Lin, Shanshan Jiang, Bin Dong, Xianpei Han, Jiawei Chen, Tianshu Wang, Le Sun

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by LREC-COLING 2024

All in One: Multi-Task Prompting for Graph Neural Networks (Extended Abstract)

Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu, Jihong Guan

专题命中 效率与部署 ：prompting(title,abstract)；分类 cs.AI、cs.LG

Comments submitted to IJCAI 2024 Sister Conferences Track. The original paper can be seen at arXiv:2307.01504

Making Pre-trained Language Models Great on Tabular Prediction

Jiahuan Yan, Bo Zheng, Hongxia Xu, Yiheng Zhu, Danny Z. Chen, Jimeng Sun, Jian Wu, Jintai Chen

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to ICLR 2024 as spotlight presentation (Notable Top 5%). OpenReview link is https://openreview.net/forum?id=anzIzGZuLi, codes will be available at https://github.com/jyansir/tp-berta

Anatomy of Neural Language Models

Majd Saleh, Stéphane Paquelet

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 36 Pages; 25 Figures; some typos and notation errors are corrected in this version

Layer-wise Regularized Dropout for Neural Language Models

Shiwen Ni, Min Yang, Ruifeng Xu, Chengming Li, Xiping Hu

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Journal ref LREC-COLING 2024

Fast Adversarial Attacks on Language Models In One GPU Minute

Vinu Sankar Sadasivan, Shoumik Saha, Gaurang Sriramanan, Priyatham Kattakinda, Atoosa Chegini, Soheil Feizi

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Representing Online Handwriting for Recognition in Large Vision-Language Models

Anastasiia Fadeeva, Philippe Schlattner, Andrii Maksai, Mark Collier, Efi Kokiopoulou, Jesse Berent, Claudiu Musat

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.AI、cs.LG

Quantized Embedding Vectors for Controllable Diffusion Language Models

Cheng Kang, Xinye Chen, Yong Hu, Daniel Novak

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Grounding Language Model with Chunking-Free In-Context Retrieval

Hongjin Qian, Zheng Liu, Kelong Mao, Yujia Zhou, Zhicheng Dou

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Beyond Extraction: Contextualising Tabular Data for Efficient Summarisation by Language Models

Uday Allu, Biddwan Ahmed, Vishesh Tripathi

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Submitted to IEEE

Parametric Feature Transfer: One-shot Federated Learning with Foundation Models

Mahdi Beitollahi, Alex Bie, Sobhan Hemati, Leo Maxime Brunswic, Xu Li, Xi Chen, Guojun Zhang

专题命中 效率与部署 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Comments 20 pages, 12 figures

Distilling Out-of-Distribution Robustness from Vision-Language Foundation Models

Andy Zhou, Jindong Wang, Yu-Xiong Wang, Haohan Wang

专题命中 效率与部署 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Comments Published in NeurIPS 2023

DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines

Arnav Singhvi, Manish Shetty, Shangyin Tan, Christopher Potts, Koushik Sen, Matei Zaharia, Omar Khattab

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Arnav*, Manish*, Shangyin* contributed equally to this work

MiniDisc: Minimal Distillation Schedule for Language Model Compression

Chen Zhang, Yang Yang, Qifan Wang, Jiahao Liu, Jingang Wang, Wei Wu, Dawei Song

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to EACL 2024. Code is available at https://github.com/GeneZC/MiniDisc

Accelerating Retrieval-Augmented Language Model Serving with Speculation

Zhihao Zhang, Alan Zhu, Lijie Yang, Yihua Xu, Lanting Li, Phitchaya Mangpo Phothilimthana, Zhihao Jia

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Preprint

ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reordering

Yakun Song, Zhuo Chen, Xiaofei Wang, Ziyang Ma, Xie Chen

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Working in progress

AccidentGPT: Large Multi-Modal Foundation Model for Traffic Accident Analysis

Kebin Wu, Wenbin Li, Xiaofei Xiao

专题命中 效率与部署 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Comments 8 pages, 2 figures

LMBot: Distilling Graph Knowledge into Language Model for Graph-less Deployment in Twitter Bot Detection

Zijian Cai, Zhaoxuan Tan, Zhenyu Lei, Zifeng Zhu, Hongrui Wang, Qinghua Zheng, Minnan Luo

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 7 figures

Language Model is a Branch Predictor for Simultaneous Machine Translation

Aoxiong Yin, Tianyun Zhong, Haoyuan Li, Siliang Tang, Zhou Zhao

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by IEEE ICASSP 2024

FP8-BERT: Post-Training Quantization for Transformer

Jianwei Li, Tianchi Zhang, Ian En-Hsu Yen, Dongkuan Xu

专题命中 效率与部署 ：post-training(title,abstract)；分类 cs.AI、cs.LG

Zoology: Measuring and Improving Recall in Efficient Language Models

Simran Arora, Sabri Eyuboglu, Aman Timalsina, Isys Johnson, Michael Poli, James Zou, Atri Rudra, Christopher Ré

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Toucan: Token-Aware Character Level Language Modeling

William Fleshman, Benjamin Van Durme

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Co-training and Co-distillation for Quality Improvement and Compression of Language Models

Hayeon Lee, Rui Hou, Jongpil Kim, Davis Liang, Hongbo Zhang, Sung Ju Hwang, Alexander Min

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Findings of EMNLP 2023

Data-Free Distillation of Language Model by Text-to-Text Transfer

Zheyuan Bai, Xinduo Liu, Hailin Hu, Tianyu Guo, Qinghua Zhang, Yunhe Wang

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Parameter-Efficient Cross-lingual Transfer of Vision and Language Models via Translation-based Alignment

Zhen Zhang, Jialu Wang, Xin Eric Wang

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Findings of EMNLP

ZeroQuant-HERO: Hardware-Enhanced Robust Optimized Post-Training Quantization Framework for W8A8 Transformers

Zhewei Yao, Reza Yazdani Aminabadi, Stephen Youn, Xiaoxia Wu, Elton Zheng, Yuxiong He

专题命中 效率与部署 ：post-training(title,abstract)；分类 cs.CL、cs.LG

Comments 8 pages, 2 figures

Orchestration of Emulator Assisted Mobile Edge Tuning for AI Foundation Models: A Multi-Agent Deep Reinforcement Learning Approach

Wenhan Yu, Terence Jie Chua, Jun Zhao

专题命中 效率与部署 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Integrating Language Models into Direct Speech Translation: An Inference-Time Solution to Control Gender Inflection

Dennis Fucci, Marco Gaido, Sara Papi, Mauro Cettolo, Matteo Negri, Luisa Bentivogli

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted at EMNLP 2023

Language Model Tokenizers Introduce Unfairness Between Languages

Aleksandar Petrov, Emanuele La Malfa, Philip H. S. Torr, Adel Bibi

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Published at NeurIPS 2023, Project webpage: https://aleksandarpetrov.github.io/tokenization-fairness, Code: https://github.com/AleksandarPetrov/tokenization-fairness

XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models

Davis Liang, Hila Gonen, Yuning Mao, Rui Hou, Naman Goyal, Marjan Ghazvininejad, Luke Zettlemoyer, Madian Khabsa

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments EMNLP 2023