语言大模型 / LLM

PEFTDebias : Capturing debiasing information using PEFTs

Sumit Agarwal, Aditya Srikanth Veerubhotla, Srijan Bansal

专题命中 指令微调 ：foundation model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments EMNLP 2023

Adapting Segment Anything Model (SAM) through Prompt-based Learning for Enhanced Protein Identification in Cryo-EM Micrographs

Fei He, Zhiyuan Yang, Mingyue Gao, Biplab Poudel, Newgin Sam Ebin Sam Dhas, Rajan Gyawali, Ashwin Dhakal, Jianlin Cheng, Dong Xu

专题命中 指令微调 ：foundation model(abstract)；prompting(abstract)；分类 cs.AI、cs.LG

Selectively Answering Ambiguous Questions

Jeremy R. Cole, Michael J. Q. Zhang, Daniel Gillick, Julian Martin Eisenschlos, Bhuwan Dhingra, Jacob Eisenstein

专题命中 指令微调 ：language model(abstract)；instruction tuning(abstract)；分类 cs.CL、cs.AI

Comments To appear in EMNLP 2023. 9 pages, 5 figures, 2 pages of appendix

Multilingual Mathematical Autoformalization

Albert Q. Jiang, Wenda Li, Mateja Jamnik

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Retrieval-Augmented Code Generation for Universal Information Extraction

Yucan Guo, Zixuan Li, Xiaolong Jin, Yantao Liu, Yutao Zeng, Wenxuan Liu, Xiang Li, Pan Yang, Long Bai, Jiafeng Guo, Xueqi Cheng

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

SkyMath: Technical Report

Liu Yang, Haihua Yang, Wenjun Cheng, Lei Lin, Chenxia Li, Yifu Chen, Lunan Liu, Jianfei Pan, Tianwen Wei, Biye Li, Liang Zhao, Lijie Wang, Bo Zhu, Guoliang Li, Xuejie Wu, Xilin Luo, Rui Hu

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

The Importance of Prompt Tuning for Automated Neuron Explanations

Justin Lee, Tuomas Oikarinen, Arjun Chatha, Keng-Chi Chang, Yilan Chen, Tsui-Wei Weng

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

NEFTune: Noisy Embeddings Improve Instruction Finetuning

Neel Jain, Ping-yeh Chiang, Yuxin Wen, John Kirchenbauer, Hong-Min Chu, Gowthami Somepalli, Brian R. Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Aniruddha Saha, Micah Goldblum, Jonas Geiping, Tom Goldstein

专题命中 指令微调 ：language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.LG

Comments 25 pages, Code is available on Github: https://github.com/neelsjain/NEFTune

Balancing Specialized and General Skills in LLMs: The Impact of Modern Tuning and Data Strategy

Zheng Zhang, Chen Zheng, Da Tang, Ke Sun, Yukun Ma, Yingtong Bu, Xun Zhou, Liang Zhao

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Fine-tuning and aligning question answering models for complex information extraction tasks

Matthias Engelbach, Dennis Klau, Felix Scheerer, Jens Drawehn, Maximilien Kintz

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at: 15th International Conference on Knowledge Discovery an Information Retrieval (KDIR 2023), part of IC3K

Toward Unified Controllable Text Generation via Regular Expression Instruction

Xin Zheng, Hongyu Lin, Xianpei Han, Le Sun

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted on IJCNLP-AACL 2023

LLaSM: Large Language and Speech Model

Yu Shu, Siwei Dong, Guangyao Chen, Wenhao Huang, Ruihua Zhang, Daochen Shi, Qiqi Xiang, Yemin Shi

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Engineering flexible machine learning systems by traversing functionally-invariant paths

Guruprasad Raghavan, Bahey Tharwat, Surya Narayanan Hari, Dhruvil Satani, Matt Thomson

专题命中 指令微调 ：language model(abstract)；foundation model(abstract)；分类 cs.AI、cs.LG

Comments 22 pages

UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understanding

Hao Feng, Zijian Wang, Jingqun Tang, Jinghui Lu, Wengang Zhou, Houqiang Li, Can Huang

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

GPT-3 Models are Few-Shot Financial Reasoners

Raul Salles de Padua, Imran Qureshi, Mustafa U. Karakaplan

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 8 figures

Journal ref CS & IT Conference Proceedings (2023), volume 13, number 12, pages 183-197

Gradient Sparsification For Masked Fine-Tuning of Transformers

James O' Neill, Sourav Dutta

专题命中 指令微调 ：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments Accepted to IJCNN 2023

Llama 2: Open Foundation and Fine-Tuned Chat Models

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, Thomas Scialom

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Can ChatGPT's Responses Boost Traditional Natural Language Processing?

Mostafa M. Amin, Erik Cambria, Björn W. Schuller

专题命中 指令微调 ：language model(abstract)；foundation model(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 2 Tables, 1 Figure

Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions

Hui Yang, Sifu Yue, Yunzhong He

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Transformer-based Vulnerability Detection in Code at EditTime: Zero-shot, Few-shot, or Fine-tuning?

Aaron Chan, Anant Kharkar, Roshanak Zilouchian Moghaddam, Yevhen Mohylevskyy, Alec Helyar, Eslam Kamal, Mohamed Elkamhawy, Neel Sundaresan

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

UFO: Unified Fact Obtaining for Commonsense Question Answering

Zhifeng Li, Yifan Fan, Bowei Zou, Yu Hong

专题命中 指令微调 ：language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments IJCNN 2023

Discrete Prompt Optimization via Constrained Generation for Zero-shot Re-ranker

Sukmin Cho, Soyeong Jeong, Jeongyeon Seo, Jong C. Park

专题命中 指令微调 ：LLM(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Findings of ACL 2023 Camera Ready

Generating coherent comic with rich story using ChatGPT and Stable Diffusion

Ze Jin, Zorina Song

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

LACoS-BLOOM: Low-rank Adaptation with Contrastive objective on 8 bits Siamese-BLOOM

Wen-Yu Hua, Brian Williams, Davood Shamsi

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

PTP: Boosting Stability and Performance of Prompt Tuning with Perturbation-Based Regularizer

Lichang Chen, Heng Huang, Minhao Cheng

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 13 pages;

Chinese Open Instruction Generalist: A Preliminary Release

Ge Zhang, Yemin Shi, Ruibo Liu, Ruibin Yuan, Yizhi Li, Siwei Dong, Yu Shu, Zhaoqun Li, Zekun Wang, Chenghua Lin, Wenhao Huang, Jie Fu

专题命中 指令微调 ：language model(abstract)；instruction tuning(abstract)；分类 cs.CL、cs.AI

In-Context Learning with Many Demonstration Examples

Mukai Li, Shansan Gong, Jiangtao Feng, Yiheng Xu, Jun Zhang, Zhiyong Wu, Lingpeng Kong

专题命中 指令微调 ：language model(abstract)；instruction tuning(abstract)；分类 cs.CL、cs.AI

Comments Preprint, under review

FedTune: A Deep Dive into Efficient Federated Fine-Tuning with Pre-trained Transformers

Jinyu Chen, Wenchao Xu, Song Guo, Junxiao Wang, Jie Zhang, Haozhao Wang

专题命中 指令微调 ：language model(abstract)；foundation model(abstract)；分类 cs.CL、cs.LG

Active Example Selection for In-Context Learning

Yiming Zhang, Shi Feng, Chenhao Tan

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2022, code is available at https://github.com/ChicagoHAI/active-example-selection

Memorization in NLP Fine-tuning Methods

Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David Evans, Taylor Berg-Kirkpatrick

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG