We propose a foundation model for soccer, which is able to predict subsequent actions in a soccer match from a given input sequence of actions. As a proof of concept, we train a transformer architecture on three seasons of data from a professional soccer league. We quantitatively and qualitatively compare the performance of this transformer architecture to two baseline models: a Markov model and a multi-layer perceptron. Additionally, we discuss potential applications of our model. We provide an open-source implementation of our methods at https://github.com/danielhocevar/Foundation-Model-for-Soccer.
Physics-Guided Multimodal Transformers are the Necessary Foundation for the Next Generation of Meteorological Science
School of Artificial Intelligence, Beijing University of Posts;Huawei Noah's Ark Lab;Department of Earth System Science, Tsinghua University;College of Computing \& Data Science, Nanyang Technological University;State Key Laboratory of Networking;Switching Technology, Beijing University of Posts
Pretrained battery transformer (PBT): A foundation model for battery life prediction
Guangzhou Municipal Key Laboratory of Materials Informatics and Sustainable Energy and Environment Thrust, The Hong Kong University of Science and Technology (Guangzhou)(广州材料信息学与可持续能源与环境方向市重点实验室,香港科技大学(广州));Department of Computer Science & Engineering, The Hong Kong University of Science and Technology(计算机科学与工程系,香港科技大学);Guangzhou Municipal Key Laboratory of Materials Informatics and Data Science and Analytics Thrust, The Hong Kong University of Science and Technology (Guangzhou)(广州材料信息学与数据科学与分析方向市重点实验室,香港科技大学(广州));Academy of Interdisciplinary Studies, The Hong Kong University of Science and Technology(交叉学科研究院,香港科技大学);Guangzhou HKUST Fok Ying Tung Research Institute(广州科技大学福 Ying Tung 研究院);Material Genome Institute, Shanghai University(材料基因组研究所,上海大学)
Department of Electronics and Computer Engineering(电子与计算机工程系);Shenzhen MSU-BIT University(深圳MSU-BIT大学);School of Computer Science and Technology(计算机科学与技术学院)
$MV_{Hybrid}$: Improving Spatial Transcriptomics Prediction with Hybrid State Space-Vision Transformer Backbone in Pathology Vision Foundation Models
AI Research Team(人工智能研究团队);AI Research Lab, Deepnoid Seoul(Deepnoid首尔人工智能研究实验室);Department of Hospital Pathology, College of Medicine, The Catholic University of Korea(韩国天主大学医学院医院病理科)
NORCE Norwegian Research Centre(挪威研究中心);Department of Mathematics, Hong Kong Baptist University(香港浸会大学数学系);School of Data Science and Department of Operations and Risk Management, Lingnan University(岭南大学数据科学学院与运营及风险管理系)
Cross-Attention Message-Passing Transformers for Code-Agnostic Decoding in 6G Networks
Institute of Artificial Intelligence, Pohang University of Science and Technology (POSTECH)(人工智能研究院,POSTECH大学);Department of Electrical, Electronic and Computer Engineering, University of Ulsan(电子电气与计算机工程系,乌山大学);Department of Electrical and Computer Engineering, Sungkyunkwan University(电气与计算机工程系,全北国立大学);Department of Electrical Engineering, Pohang University of Science and Technology (POSTECH)(电气工程系,POSTECH大学);Department of Electrical and Computer Engineering, Seoul National University(电气与计算机工程系,首尔国立大学)
BiPFT: Binary Pre-trained Foundation Transformer with Low-rank Estimation of Binarization Residual Polynomials
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Beijing Academy of Artificial Intelligence(北京智源人工智能研究院);Beihang University(北京航空航天大学)
MERIT: Scaling KG Foundation Models with Relation Transformers
Institute for Artificial Intelligence, University of Stuttgart(斯图加特大学人工智能研究所);School of Informatics, University of Edinburgh(爱丁堡大学信息学院);University of Southampton(南安普顿大学)
MEG-GPT: A transformer-based foundation model for magnetoencephalography data
Oxford Centre for Integrative Neuroimaging (OxCIN)(牛津整合神经影像中心);University of Oxford(牛津大学);Department of Psychiatry(精神病学系);Nuffield Department of Clinical Neurosciences(努尔菲尔德临床神经科学系);Department of Engineering Science(工程科学系)
本文提出 MEG-GPT,一种基于 transformer、时间注意力和下一时间点预测的 MEG 基础模型,并通过无损数据驱动分词器实现真实脑信号生成与跨会话、跨受试者零样本解码提升。
TAFFY: A Task-Adaptive Tabular Foundation Model with In-Context Diversity
Carnegie Mellon University(卡内基梅隆大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Guangdong University of Technology(广东工业大学);University of California, San Diego(加利福尼亚大学圣迭戈分校)
RmGPT: A Foundation Model with Generative Pre-trained Transformer for Fault Diagnosis and Prognosis in Rotating Machinery
School of Aeronautics and Astronautics, Shanghai Jiao Tong University(航空宇航学院,上海交通大学);Shanghai Innovation Institute(上海创新研究院);MINES Paris-PSL University(巴黎-里昂商学院);Energy Department, Politecnico di Milano(能源部门,米兰理工学院);Beijing Aeronautical Technology Research Center(北京航空航天科技研究所)
DT-Transformer: A Foundation Model for Disease Trajectory Prediction on a Real-world Health System
Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School(药理流行病学与药效学部,医学部,布里奇沃特医院,哈佛医学院);Harvard T.H. Chan School of Public Health, Harvard University(哈佛大学T.H. 陈公共卫生学院);Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University(自然与人工智能研究学院,哈佛大学);Broad Institute of MIT and Harvard, Cambridge, MA, USA(MIT与哈佛大学Broad研究所,剑桥,马萨诸塞州,美国);Harvard Data Science Initiative, Harvard University(哈佛大学数据科学计划)
Radio Foundation Models: Pre-training Transformers for 5G-based Indoor Localization
Fraunhofer Institute for Integrated Circuits (IIS)(弗劳恩霍夫集成电路研究所);Faculty of Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg(弗里德里希-亚历山大-埃尔兰根-纽伦堡大学工程学院)
QuantFlow: A Federated Mamba-Based Post-Transformer Foundation Model for Time-Series Forecasting
Department of Computer Science and Engineering, University of Science and Technology Chittagong(信息科学与工程系,查塔姆冈科技大学);Department of Electrical and Electronic Engineering, University of Science and Technology Chittagong(电气电子工程系,查塔姆冈科技大学);Faculty of Science, Engineering and Technology, University of Science and Technology Chittagong(科学、工程与技术学院,查塔姆冈科技大学)
Scaling Generative Foundation Models for Chest Radiography with Rectified Flow Transformers
Imperial College London(帝国理工学院);Causality in Healthcare AI Hub(医疗AI因果关系中心);University of Edinburgh(爱丁堡大学);Cleveland Clinic London(克利夫兰诊所伦敦);Department of Perioperative Medicine, CHU Clermont-Ferrand(克莱蒙费朗大学医院围手术期医学科);Department of Medicine, Massachusetts General Hospital(麻省总医院医学部);Broad Institute of MIT and Harvard(麻省理工学院与哈佛大学博德研究所)