Preference optimization is a standard approach to fine-tuning large language models to align with human preferences. The quantity, diversity, and representativeness of the preference dataset are critical to the effectiveness of preference optimization. However, obtaining a large amount of preference annotations is difficult in many applications. This raises the question of how to use the limited annotation budget to create an effective preference dataset. To this end, we propose Annotation-Efficient Preference Optimization (AEPO). Instead of exhaustively annotating preference over all available response texts, AEPO selects a subset of responses that maximizes diversity and representativeness from the available responses and then annotates preference over the selected ones. In this way, AEPO focuses the annotation budget on labeling preferences over a smaller but informative subset of responses. We evaluate the performance of preference learning using AEPO on three datasets and show that it outperforms the baselines with the same annotation budget. Our code is available at https://github.com/CyberAgentAILab/annotation-efficient-po
State Key Laboratory of AI Safety（人工智能安全国家重点实验室）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models
University of Oxford（牛津大学）；University of Pennsylvania（宾夕法尼亚大学）；Bocconi University（博科尼大学）；ML Commons；Contextual AI；Factored AI；UCL（伦敦大学学院）；Cohere；MetaAI；New York University（纽约大学）；AWS AI Labs（亚马逊云科技AI实验室）；Meedan
Zurich University of Applied Sciences (ZHAW)（苏黎世应用科学大学（ZHAW））；National Distance Education University (UNED)（国立远程教育大学（UNED））；University of the Basque Country (UPV/EHU)（巴斯克大学（UPV/EHU））
In this paper, we introduce a novel methodology to efficiently construct a corpus for question answering over structured data. For this, we introduce an intermediate representation that is based on the logical query plan in a database called Operation Trees (OT). This representation allows us to invert the annotation process without losing flexibility in the types of queries that we generate. Furthermore, it allows for fine-grained alignment of query tokens to OT operations. In our method, we randomly generate OTs from a context-free grammar. Afterwards, annotators have to write the appropriat…
RiskNet: A large-scale dataset of AI risk incidents from news with alignment and multi-dimensional annotations
Beijing University of Posts and Telecommunications（北京邮电大学）；Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance（多模态数据智能感知与治理北京市重点实验室）
Exploring LLM-based Data Annotation Strategies for Medical Dialogue Preference Alignment
School of Computer Science, Peking University（北京大学计算机学院）；Key Laboratory of High Confidence Software Technologies(PKU), MOE, China（教育部高可信软件技术重点实验室（北京大学））；Beijing Jiaotong University（北京交通大学）
In machine learning the best performance on a certain task is achieved by fully supervised methods when perfect ground truth labels are available. However, labels are often noisy, especially in remote sensing where manually curated public datasets are rare. We study the multi-modal cadaster map alignment problem for which available annotations are mis-aligned polygons, resulting in noisy supervision. We subsequently set up a multiple-rounds training scheme which corrects the ground truth annotations at each round to better train the model at the next round. We show that it is possible to reduc…
Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement
Key Laboratory of Social Computing and Cognitive Intelligence, Dalian University of Technology（大连理工大学社会计算与认知智能重点实验室）；School of Computer Science and Technology, Xinjiang Normal University（新疆师范大学计算机科学与技术学院）；Social AI Studio, Singapore University of Technology and Design（新加坡科技设计大学社会AI工作室）
Dataset-Aware Cold-Start Active Learning for Annotation-Efficient 3D Medical Image Segmentation
IADI (U1254)（IADI（U1254））；Inserm and Université de Lorraine（Inserm和洛林大学）；CIC-IT 1433；Inserm, Université de Lorraine（Inserm和洛林大学）；CHRU Nancy（南锡大学医院）；Université de Lorraine（洛林大学）；CHRU-Nancy（南锡大学医院）；Pôle de la femme（妇女中心）
Refining and Reusing Annotation Guidelines for LLM Annotation
The Graduate University for Advanced Studies, SOKENDAI（Graduate University for Advanced Studies, SOKENDAI）；National Institute of Informatics (NII)（National Institute of Informatics）；BioData Science Initiative (BSI), National Institute of Genetics (NIG)（BioData Science Initiative (BSI), National Institute of Genetics）
The medical imaging literature has witnessed remarkable progress in high-performing segmentation models based on convolutional neural networks. Despite the new performance highs, the recent advanced segmentation models still require large, representative, and high quality annotated datasets. However, rarely do we have a perfect training dataset, particularly in the field of medical imaging, where data and annotations are both expensive to acquire. Recently, a large body of research has studied the problem of medical image segmentation with imperfect datasets, tackling two major dataset limitat…
VA Boston Healthcare System（波士顿退伍军人医疗保健系统）；Boston University School of Medicine（波士顿大学医学院）；University of Vermont（佛蒙特大学）
Sequence alignments are used to capture patterns composed of elements representing multiple conceptual levels through the alignment of sequences that contain overlapping and variable length annotations. The alignments also determine the proper context window of words and phrases that most directly impact the meaning of a given target within a sentence, eliminating the need to predefine a fixed context window of words surrounding the targets. We evaluated the system using the CoNLL-2003 named entity recognition (NER) task.
Summarization is a challenging problem, and even more challenging is to manually create, correct, and evaluate the summaries. The severity of the problem grows when the inputs are multi-party dialogues in a meeting setup. To facilitate the research in this area, we present ALIGNMEET, a comprehensive tool for meeting annotation, alignment, and evaluation. The tool aims to provide an efficient and clear interface for fast annotation while mitigating the risk of introducing errors. Moreover, we add an evaluation mode that enables a comprehensive quality evaluation of meeting minutes. To the best …
Annotating large collections of textual data can be time consuming and expensive. That is why the ability to train models with limited annotation budgets is of great importance. In this context, it has been shown that under tight annotation budgets the choice of data representation is key. The goal of this paper is to better understand why this is so. With this goal in mind, we propose a metric that measures the extent to which a given representation is structurally aligned with a task. We conduct experiments on several text classification datasets testing a variety of models and representatio…
University of Copenhagen（哥本哈根大学）；Carnegie Mellon University（卡内基梅隆大学）
Neuroscientists evaluate deep neural networks for natural language processing as possible candidate models for how language is processed in the brain. These models are often trained without explicit linguistic supervision, but have been shown to learn some linguistic structure in the absence of such supervision (Manning et al., 2020), potentially questioning the relevance of symbolic linguistic theories in modeling such cognitive processes (Warstadt and Bowman, 2020). We evaluate across two fMRI datasets whether language models align better with brain recordings, if their attention is biased b…
University of Pennsylvania（宾夕法尼亚大学）；IBM Research（IBM研究院）
We develop high performance multilingualAbstract Meaning Representation (AMR) sys-tems by projecting English AMR annotationsto other languages with weak supervision. Weachieve this goal by bootstrapping transformer-based multilingual word embeddings, in partic-ular those from cross-lingual RoBERTa (XLM-R large). We develop a novel technique forforeign-text-to-English AMR alignment, usingthe contextual word alignment between En-glish and foreign language tokens. This wordalignment is weakly supervised and relies onthe contextualized XLM-R word embeddings.We achieve a highly competitive performa…
