We introduce LAMPO, a novel paradigm that leverages Large Language Models (LLMs) for solving few-shot multi-class ordinal classification tasks. Unlike conventional methods, which concatenate all demonstration examples with the test instance and prompt LLMs to produce the pointwise prediction, our framework uses the LLM as a preference machine that makes a relative comparative decision between the test instance and each demonstration. A self-supervised method is then introduced to aggregate these binary comparisons into the final ordinal decision. LAMPO addresses several limitations inherent in previous methods, including context length constraints, ordering biases, and challenges associated with absolute point-wise estimation. Extensive experiments on seven public datasets demonstrate LAMPO's remarkably competitive performance across a diverse spectrum of applications (e.g., movie review analysis and hate speech detection). Notably, in certain applications, the improvement can be substantial, exceeding 20% in an absolute term. Moreover, we believe LAMPO represents an interesting addition to the non-parametric application layered on top of LLMs, as it supports black-box LLMs without necessitating the outputting of LLM's internal states (e.g., embeddings), as seen in previous approaches.
Learning Ordinal Probabilistic Reward from Preferences
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Ritzz-AI；Shenzhen University of Advanced Technology（深圳先进技术大学）
Mitigating Hallucination Through Theory-Consistent Symmetric Multimodal Preference Optimization
Shandong University（山东大学）；Southern University of Science and Technology（南方科技大学）；University of Georgia（佐治亚大学）；National University of Singapore（新加坡国立大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
HEAL: A Hypothesis-Based Preference-Aware Analysis Framework
School of Computer Science and Engineering, Northeastern University（计算机科学与工程学院，东北大学）；NiuTrans Research（NiuTrans研究院）；CAS Key Laboratory of Behavioral Science, Institute of Psychology, CAS（行为科学重点实验室，心理学研究所）
PREF-XAI: Preference-Based Personalized Rule Explanations of Black-Box Machine Learning Models
University of Catania, Department of Economics and Business（卡塔尼亚大学经济与商业系）；Poznan University of Technology, Institute of Computing Science（波兹南技术大学计算科学研究所）；Systems Research Institute, Polish Academy of Sciences（波兰科学院系统研究所）
