The proliferation of deep learning in natural language processing (NLP) has led to the development and release of innovative technologies capable of understanding and generating human language with remarkable proficiency. Atinuke, a Transformer-based neural network, optimises performance across various language tasks by utilising a unique configuration. The architecture interweaves layers for processing sequential data with attention mechanisms to draw meaningful affinities between inputs and outputs. Due to the configuration of its topology and hyperparameter tuning, it can emulate human-like language by extracting features and learning complex mappings. Atinuke is modular, extensible, and integrates seamlessly with existing machine learning pipelines. Advanced matrix operations like softmax, embeddings, and multi-head attention enable nuanced handling of textual, acoustic, and visual signals. By unifying modern deep learning techniques with software design principles and mathematical theory, the system achieves state-of-the-art results on natural language tasks whilst remaining interpretable and robust.
Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Hong Kong Baptist Univeristy(香港浸会大学);Harbin Institude of Technology, Shenzhen(哈尔滨工业大学(深圳))
Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models
Nanjing University(南京大学);Tsinghua University(清华大学);Hong Kong Polytechnic University(香港理工大学);Johns Hopkins University(约翰斯·霍普金斯大学);New York University(纽约大学);NVIDIA(英伟达)
AutoBERT-Zero: Evolving BERT Backbone from Scratch
The University of Hong Kong(香港大学);Hong Kong University of Science and Technology(香港科技大学);The Education University of Hong Kong(香港教育大学);Sun Yat-sen University(中山大学)
ConsistentChat: Building Skeleton-Guided Consistent Multi-Turn Dialogues for Large Language Models from Scratch
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences, Beijing, China(中国科学院软件研究所中文信息处理实验室);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学)
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
Microsoft(微软);Singapore University of Technology and Design(新加坡科技设计大学);Peking University(北京大学);Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Chinese University of Hong Kong(香港中文大学);Tsinghua University(清华大学)
Learning structures of the French clinical language:development and validation of word embedding models using 21 million clinical reports from electronic health records
Assistance Publique – Hôpitaux de Paris(巴黎公共援助医院);Sorbonne Université(索邦大学);Inserm(法国国家健康与医学研究院);Université Sorbonne Paris Nord(索邦巴黎北大学);Centre de Recherche des Cordeliers(科德利埃研究中心);INRIA(法国国家信息与自动化研究所)
From scratch to silver: Creating trustworthy training data for patent-SDG classification using Large Language Models
Department of Industrial Engineering, Polytechnic University of Turin(工业工程系,都灵理工学院);Venice School of Management, Ca Foscari University of Venice(威尼斯管理学院,威尼斯福斯卡里大学)
ChakmaNMT: Machine Translation for a Low-Resource and Endangered Language via Transliteration
University of Arizona(亚利桑那大学);Bangladesh University of Engineering and Technology(孟加拉工程与技术大学);University of Rochester(罗切斯特大学);Chittagong University of Engineering and Technology(丘比特工程与技术大学)
LAMPER: LanguAge Model and Prompt EngineeRing for zero-shot time series classification
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学清华大学深圳国际研究生院);School of Automation, Wuhan University of Technology(武汉理工大学自动化学院)
CMAP, École Polytechnique, Institut Polytechnique de Paris(巴黎综合理工学院应用数学中心,巴黎理工学院);Samovar, Télécom SudParis, département CITI, TIPIC, Institut Polytechnique de Paris(萨莫瓦尔实验室,南巴黎电信学院,CITI系,TIPIC实验室,巴黎理工学院);Deep Mind(深度思维);Google Research, Brain Team(谷歌研究院大脑团队)
本文介绍了TRUNCATED REINFORCENT LEARNING FOR LANGUAGE(TrufLL),这是一种仅使用强化学习(RL)从头开始训练条件语言模型的新颖方法。由于RL方法不能成功地扩展到大的动作空间,我们使用通用的语言模型动态截断词汇空间。因此,TrufLL能够训练语言代理,只与其环境交互,没有任何特定于任务的先验知识;它只与任务无关的语言模型指导。有趣的是,这种方法避免了对标记数据集的依赖,并从本质上减少了预先训练的政策缺陷,如语言或暴露偏差。我们评估TrufLL的两个视觉问题生成任务,我们报告的性能和语言指标,这wethen证实了人类的评价积极的结果。据我们所知,这是第一种(几乎)从零开始成功学习语言生成策略的方法。