While recent advancements in foundation models have significantly impacted machine learning, rigorous tests on the performance of time series foundation models (TSFMs) remain largely underexplored. This paper presents an empirical study evaluating the zero-shot, long-horizon forecasting abilities of several leading TSFMs over two synthetic datasets constituting noisy periodic time series. We assess model efficacy across different noise levels, underlying frequencies, and sampling rates. As benchmarks for comparison, we choose two statistical techniques: a Fourier transform (FFT)-based approach and a linear autoregressive (AR) model. Our findings demonstrate that while for time series with bounded periods and higher sampling rates, TSFMs can match or outperform the statistical approaches, their forecasting abilities deteriorate with longer periods, higher noise levels, lower sampling rates and more complex shapes of the time series.
Evaluating Temporal Plasticity in Foundation Time Series Models for Incremental Fine-tuning
School of Software Engineering(软件工程学院);National Key Laboratory of Fundamental Science of Synthetic Vision(合成视觉基础科学国家重点实验室);School of Mechanical Engineering(机械工程学院);China Electric Power Research Institute(中国电力科学研究院);School of Computer Science(计算机科学学院)
TabReliBench: Evaluating Tabular Foundation Models Beyond Predictive Accuracy
Renmin University of China(中国人民大学);Cerebras Systems, Inc.(赛博拉斯系统公司);National University of Singapore(新加坡国立大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
EvalBlocks: A Modular Pipeline for Rapidly Evaluating Foundation Models in Medical Imaging
Radboud University Medical Center(拉德堡德大学医学中心);Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands(荷兰奈梅亨拉德堡德大学医学中心诊断图像分析组)
Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicine
Harvard Medical School(哈佛医学院);Stanford University(斯坦福大学);Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University(哈佛大学约翰·A·保尔森工程与应用科学学院)
Stanford Sleep Bench: Evaluating Polysomnography Pre-training Methods for Sleep Foundation Models
Stanford University(斯坦福大学);Technical University of Denmark(技术大学);Danish Center for Sleep Medicine(丹麦睡眠医学中心);University of Copenhagen(哥本哈根大学);Harvard Medical School(哈佛医学院)
Evaluating the Generalizability of Foundation Models for Extreme Environmental Events: Case Study of California Wildfire PM2.5
College of Engineering, University of Georgia(佐治亚大学工程学院);Information System Department, UMBC(马里兰大学巴尔的摩郡分校信息系统系);College of Graduate and Professional Studies, Trine University(特瑞大学研究生与专业研究学院);School of Professional Studies, Columbia University(哥伦比亚大学专业研究学院)
Mars-Bench: A Benchmark for Evaluating Foundation Models for Mars Science Tasks
School of Computing and Augmented Intelligence, Arizona State University(计算与增强智能学院,亚利桑那州立大学);School of Earth and Space Exploration, Arizona State University(地球与空间探索学院,亚利桑那州立大学);Jet Propulsion Laboratory, California Institute of Technology(喷气推进实验室,加州理工学院)
BadLabel: A Robust Perspective on Evaluating and Enhancing Label-noise Learning
the University of Auckland(奥克兰大学);the University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);RIKEN AIP(理化学研究所人工智能项目);Hong Kong Baptist University(香港浸会大学);the University of Sydney(悉尼大学);Shandong University(山东大学);Shandong Research Institute of Industrial Technology(山东工业技术研究院);the University of Tokyo(东京大学)
Impact of Noisy Supervision in Foundation Model Learning
Carnegie Mellon University(卡内基梅隆大学);Microsoft Research Asia(微软亚洲研究院);Southern University of Science and Technology(南方科技大学);RIKEN(理化学研究所);the University of Tokyo(东京大学);William & Mary(威廉玛丽学院)
Evaluating Time-Series Foundation Models and Multimodal Dietary Context for CGM Forecasting
University of California, Los Angeles(加州大学洛杉矶分校);Union County Magnet High School(联合县磁石高中);Huntington Beach High School(亨廷顿海滩高中);Crean Lutheran High School(克林路德高中);Portola High School(波托拉高中);Tsinghua University(清华大学)
Evaluating AlphaEarth Foundations Embeddings for Wildfire Susceptibility Mapping
UNSW Sydney(新南威尔士大学悉尼分校);ARC Centre of Excellence for the Weather of the 21st Century(21世纪天气ARC卓越中心);Climate Change Research Centre, University of New South Wales(新南威尔士大学气候变化研究中心);UNSW AI Institute(新南威尔士大学人工智能研究院);Wisconsin School of Business, University of Wisconsin–Madison(威斯康星大学麦迪逊分校威斯康星商学院)
TaxDistill: Improving Metagenomic Taxonomic Annotation via Distilled Genomic Foundation Models
National Genomics Data Center, China National Center for Bioinformation(中国生物信息中心国家基因组数据中心);Beijing Key Laboratory of Intelligent Governance and Application of Biological Big Data, China National Center for Bioinformation(北京生物大数据智能治理与应用重点实验室,中国生物信息中心);Beijing Institute of Genomics, Chinese Academy of Sciences(北京基因组研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学)
Evaluating Computational Pathology Foundation Models for Prostate Cancer Grading under Distribution Shifts
Department of Medical Epidemiology and Biostatistics, Karolinska Institutet(卡罗林斯卡研究所医学流行病学与生物统计学系);Department of Engineering Science, University of Oxford(牛津大学工程科学系)
Evaluating Precise Geolocation Inference Capabilities of Vision Language Models
Apart Research(阿帕特研究院);University of Maryland(马里兰大学);Hanoi University of Science and Technology(河内科技大学);University of Science and Technology of Hanoi(河内科学技术大学)
本文构建 Google Street View 全球分布基准,评估基础 VLM 及可调用工具的智能体在单图像地理定位上的能力,揭示其低误差表现及在线隐私风险。