本文提出指令嵌入概念及基准 IEB，并设计 PIE 方法使表示聚焦任务而非语义，实验证明其在任务识别和下游指令任务中表现优越。
摘要：
指令数据对于提升大语言模型（LLMs）达到人类水平表现的能力至关重要。最近的研究 LIMA 表明，对齐本质上是一个模型利用预训练知识和技能，调整指令的交互风格或格式以解决各种任务的过程。因此，对于指令数据而言，最重要的方面是它所代表的任务，而不是具体的语义和知识信息。指令的潜在表示在一些与指令相关的任务（如数据选择和演示检索）中发挥作用。然而，这些表示通常来源于文本嵌入，包含了影响任务类别表示的总体语义信息。在这项工作中，我们引入了一个新概念——指令嵌入，并构建了指令嵌入基准（Instruction Embedding Benchmark，IEB）用于其训练和评估。随后，我们提出了一种基线方法——基于提示的指令嵌入（Prompt-based Instruction Embedding，PIE），使表示更加关注任务。PIE 与其他嵌入方法在 IEB 上通过两个设计任务进行的评估表明，其在准确识别任务类别方面具有优越性能。此外，指令嵌入在四个下游任务中的应用展示了其在指令相关任务中的有效性和适用性。
英文摘要：
Instruction data is crucial for improving the capability of Large Language Models (LLMs) to align with human-level performance. Recent research LIMA demonstrates that alignment is essentially a process where the model adapts instructions' interaction style or format to solve various tasks, leveraging pre-trained knowledge and skills. Therefore, for instructional data, the most important aspect is the task it represents, rather than the specific semantics and knowledge information. The latent representations of instructions play roles for some instruction-related tasks like data selection and demonstrations retrieval. However, they are always derived from text embeddings, encompass overall semantic information that influences the representation of task categories. In this work, we introduce a new concept, instruction embedding, and construct Instruction Embedding Benchmark (IEB) for its training and evaluation. Then, we propose a baseline Prompt-based Instruction Embedding (PIE) method to make the representations more attention on tasks. The evaluation of PIE, alongside other embedding methods on IEB with two designed tasks, demonstrates its superior performance in accurately identifying task categories. Moreover, the application of instruction embeddings in four downstream tasks showcases its effectiveness and suitability for instruction-related tasks.
ImpRIF: Stronger Implicit Reasoning Leads to Better Complex Instruction Following
ByteDance China（字节跳动中国）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）
MLLMReID: Multimodal Large Language Model-based Person Re-identification
School of Computer Science and Engineering（计算机科学与工程学院）；Beihang University（北京航空航天大学）；State Key Laboratory of Virtual Reality Technology and Systems（虚拟现实技术与系统国家重点实验室）
GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI
Shandong University（山东大学）；National University of Singapore（新加坡国立大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Xiaomi Corporation（小米公司）
LangForce: Bayesian Decomposition of Vision Language Action Models via Latent Action Queries
Huazhong University of Science and Technology（华中科技大学）；Beijing Zhongguancun Academy（北京中关村学院）；Zhongguancun Institute of Artificial Intelligence（中关村人工智能研究院）；Harbin Institute of Technology（哈尔滨工业大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Zhengzhou University（郑州大学）；Beihang University（北航）；East China Normal University（东华大学）；DeepCybot Co., Ltd.（DeepCybot有限公司）
Robust Text-driven Image Editing Method that Adaptively Explores Directions in Latent Spaces of StyleGAN and CLIP
Police Info-communications Research Center, National Police Academy（警察大学校警察信息通信研究中心）；NTT Human Informatics Laboratories, NTT Corporation（日本电报电话公司NTT人类信息学实验室）
