This paper presents LLM-jp-4, a family of Japanese sovereign large language models (LLMs) developed with an emphasis on transparency, reproducibility, and practical usability. LLM-jp-4 comprises an 8B dense model and a 32B-A3B mixture-of-experts model, both trained from scratch on approximately 12 trillion tokens. To support independent verification and reproduction, we establish explicit criteria for selecting training data, external models, and software, and make the development process publicly available. Evaluation results demonstrate substantial improvements over previous LLM-jp models and competitive performance against open-weight models of similar scale. By documenting the development principles, training procedures, and empirical findings of LLM-jp-4, we aim to provide a reference for future sovereign AI development.
Assessing Socio-Cultural Alignment and Technical Safety of Sovereign LLMs
Graduate School of Data Science, Dept. of Data Science, Seoul National University(数据科学研究生院、数据科学系、首尔国立大学);College of Engineering, Dept. of Computer Science and Engineering, Seoul National University(工程学院、计算机科学与工程系、首尔国立大学)
Evaluating LLM-Based Translation of a Low-Resource Technical Language: The Medical and Philosophical Greek of Galen
Department of Classical and Mediterranean Studies, Vanderbilt University(维斯尼尔大学古典与地中海研究系);Center for Hellenic Studies, Harvard University(哈佛大学希腊研究中心);Department of Philosophy, Vanderbilt University(维斯尼尔大学哲学系);Machine Intelligence and Normative Theory (MINT) Lab, Australian National University(澳大利亚国立大学机器智能与规范理论实验室);Department of the Classics, Harvard University(哈佛大学古典学系)
Red Teaming LLMs as Socio-Technical Practice: From Exploration and Data Creation to Evaluation
Responsible Tech Research IBM Research Yorktown Heights New York USA(负责技术研究 IBM 研究 Yorktown Heights 新 York 美国);IBM Research(IBM 研究);The Pennsylvania State University(宾夕法尼亚州立大学);University of Notre Dame(诺特达美大学)
A Comparative Study of Student Perspectives on Technical Writing Feedback Quality: Evaluating LLMs, SLMs, and Humans in Computer Science Topics
McMaster University(麦斯特大学);Department of Computer Science, University of Toronto(多伦多大学计算机科学系);Research Institute for the Study of University Pedagogy, University of Toronto Mississauga(多伦多大学密西根分校大学教学研究学院);Department of Mathematical and Computational Sciences, University of Toronto Mississauga(多伦多大学密西根分校数学与计算科学系)
Adapting Technical-Service LLM Agents with Latent Logic Augmentation, Robust Noise Reduction, and Hybrid Reward Modeling
School of Mathematics and Sciences, Fudan University, Shanghai, China(复旦大学数学科学学院,上海,中国);Shanghai Center for Mathematical Sciences, Fudan University, Shanghai, China(复旦大学上海数学中心,上海,中国);Alibaba Group, Hangzhou, China(阿里巴巴集团,杭州,中国);Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China(脑启发式智能科学技术研究院,复旦大学,上海,中国);Center for Applied Mathematics & Shanghai Key Laboratory of Contemporary Applied Mathematics, Fudan University, Shanghai, China(应用数学中心及上海当代应用数学重点实验室,复旦大学,上海,中国)
Human Preferences in Large Language Model Latent Space: A Technical Analysis on the Reliability of Synthetic Data in Voting Outcome Prediction
Ludwig-Maximilian-University Munich(慕尼黑大学);Munich Center for Machine Learning(慕尼黑机器学习中心);University of Bayreuth(拜罗伊特大学);Fraunhofer Institute for Applied Information Technology FIT(弗劳恩霍夫应用信息技术研究所);University of Maryland(马里兰大学)
Grammar Induction for Minimalist Grammars using Variational Bayesian Inference : A Technical Report
Stanford University(斯坦福大学);McGill University(麦吉尔大学);École Polytechnique Fédérale de Lausanne(洛桑联邦理工学院);University of California, San Diego(加利福尼亚大学圣地亚哥分校)
Technical report on Conversational Question Answering
ZhuiYi Technology(追一科技)
会话问答是一项具有挑战性的任务,因为它需要了解会话历史。在这个项目中,我们提出了一个新的系统RoBERTa + AT +KD,它涉及理性标记多任务,对抗训练,知识蒸馏和语言后处理策略。在没有数据增强的情况下,我们的单一模型在CoQA测试集上达到90.4(F1),比当前最先进的单一模型高出2.6%F1。