Extractive summarization can produce faithful summaries but often requires additional constraints such as a desired summary length. Traditional sentence compression models do not typically consider the constraints because of their restricted model abilities, which require model modifications for coping with them. To bridge this gap, we propose Instruction-based Compression (InstructCMP), an approach to the sentence compression task that can consider the length constraint through instructions by leveraging the zero-shot task-solving abilities of Large Language Models (LLMs). For this purpose, we created new evaluation datasets by transforming traditional sentence compression datasets into an instruction format. By using the datasets, we first reveal that the current LLMs still face challenges in accurately controlling the length for a compressed text. To address this issue, we propose an approach named "length priming," that incorporates additional length information into the instructions without external resources. While the length priming effectively works in a zero-shot setting, a training dataset with the instructions would further improve the ability of length control. Thus, we additionally created a training dataset in an instruction format to fine-tune the model on it. Experimental results and analysis show that applying the length priming significantly improves performances of InstructCMP in both zero-shot and fine-tuning settings without the need of any model modifications.
Jailbreak Instruction-Tuned LLMs via end-of-sentence MLP Re-weighting
Peking University(北京大学);School of Science(理学院);Westlake University(西湖大学);Beijing International Center for Mathematical Research(北京国际数学研究中心);Center for Machine Learning Research(机器学习研究中心)
Data-CUBE: Data Curriculum for Instruction-based Sentence Representation Learning
School of Information, Renmin University of China(中国人民大学信息学院);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学歌华人工智能学院);Alibaba Group(阿里巴巴集团)
University of Southern California(南加州大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);The Pennsylvania State University(宾夕法尼亚州立大学);University of California, Davis(加利福尼亚大学戴维斯分校)
Fine-Grained and Multi-Dimensional Metrics for Document-Level Machine Translation
Digital Twin Institute, Eastern Institute of Technology, Ningbo, China(数字孪生研究所,东部技术研究所,宁波,中国);Saarland University(萨尔兰州大学);Department of Computing, The Hong Kong Polytechnic University(计算系,香港理工大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
School of Computer Science and Informatics, Cardiff University(卡迪夫大学计算机科学与信息学院);Faculty of Computing and Information Technology, King Abdulaziz University(阿卜杜勒阿齐兹大学计算与信息技术学院)
Human Values in a Single Sentence: Moral Presence, Hierarchies, and Transformer Ensembles on the Schwartz Continuum
PRHLT Research Center, Universitat Politècnica de València(PRHLT研究中心,巴塞罗那理工大学);School of Science, Engineering and Design, Universidad Europea de Valencia(科学、工程与设计学院,欧洲大学瓦伦西亚分校);Valencian Graduate School(瓦伦西亚研究生学院);Research Network of Artificial Intelligence (ValgrAI)(人工智能研究网络(ValgrAI))
University of Southern California(南加州大学);University of Chicago(芝加哥大学);Temple University(天普大学);University of Birmingham(伯明翰大学);Toyota Technological Institute at Chicago(芝加哥丰田技术研究院);Penn State University(宾夕法尼亚州立大学)
Do Schwartz Higher-Order Values Help Sentence-Level Human Value Detection? A Study of Hierarchical Gating and Calibration
PRHLT Research Center, Universitat Politècnica de València(PRHLT研究中心,瓦伦西亚理工大学);School of Science, Engineering and Design, Universidad Europea de Valencia(瓦伦西亚欧洲大学科学、工程与设计学院);Valencian Graduate School and Research Network of Artificial Intelligence (ValgrAI)(瓦伦西亚人工智能研究生院与研究网络(ValgrAI))
Enhancing Document-level Translation of Large Language Model via Translation Mixed-instructions
The Key Laboratory of Linguistic and Cultural Computing of Ministry of Education(教育部语言与文化计算重点实验室);Institute of Artificial Intelligence(人工智能研究所);Northwest Minzu University(西北民族大学);Soochow University(苏州大学)
A Hybrid Supervised-LLM Pipeline for Actionable Suggestion Mining in Unstructured Customer Reviews
Department of Computer Science & Information Systems, Birla Institute of Technology and Science, Pilani, India(印度比拉理工学院计算机科学与信息系统系);Birdeye Inc.(Birdeye公司)
Improving In-context Learning of Multilingual Generative Language Models with Cross-lingual Alignment
State Key Laboratory of Multimodal Artificial Intelligence Systems(多模态人工智能系统国家重点实验室);Institute of Automation, CAS(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)