Improving text classification with vectors of reduced precision

Training with reduced precision of a support vector machine model for text classification

This paper presents the impact of using quantization on the efficiency of multi-class text classification in the training process of a support vector machine (SVM). This work is focused on comparing the efficiency of SVM model trained using reduced precision with its original form. The main advantage of using quantization is decrease in computation time and in memory footprint on the dedicated hardware platform which supports low precision computation like GPU (16-bit) or FPGA (any bit-width). The paper presents the impact of a precision reduction of the SVM training process on text classifica…

OpenLID-v3：提高近亲语言识别精度的经验报告

OpenLID-v3: Improving the Precision of Closely Related Language Identification -- An Experience Report

针对现有语言识别工具对近亲语言和噪声区分困难的问题，通过增加训练数据、合并问题语言变体簇和引入噪声标签扩展OpenLID分类器，提出OpenLID-v3，在多个基准上提升精度。

RECSIP：分数重复聚类以提升精确率

RECSIP: REpeated Clustering of Scores Improving the Precision

针对 LLM 响应不可靠的问题，本文提出 RECSIP 框架，通过并行询问多模型并对响应评分、聚类来提升精确率，在 MMLU-Pro 上较最佳单模型提高 5.8 个百分点。

Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers

Pre-trained Transformer models such as BERT have shown great success in a wide range of applications, but at the cost of substantial increases in model complexity. Quantization-aware training (QAT) is a promising method to lower the implementation cost and energy consumption. However, aggressive quantization below 2-bit causes considerable accuracy degradation due to unstable convergence, especially when the downstream dataset is not abundant. This work proposes a proactive knowledge distillation method called Teacher Intervention (TI) for fast converging QAT of ultra-low precision pre-trained…

玩文字游戏，用奖励改进：训练语言模型进行创意联想

Playing with Words, Improving with Rewards: Training Language Models for Creative Association

本研究通过强化学习与可验证奖励（RLVR）在Codenames游戏上训练LLM，探索了规模依赖的精确度-创造力权衡，发现8B模型在保持推理能力的同时提升创造力，而小模型则牺牲创造力换取推理精度。

提高语言模型的多样性：当Temperature失效时，改变Loss

Improving Diversity in Language Models: When Temperature Fails, Change the Loss

针对提高temperature无法有效提升语言模型多样性的问题，提出利用Precision-Recall框架重新思考loss functions，在Precision和Recall之间实现了比temperature scaling更好的权衡。